CBLite .NET SDK SymmetricKey's PBKDF2 implementation

It seems as though the SymmetricKey class is using PBKDF2-SHA1 instead of PBKDF2-SHA256. This creates issues when attempting to load encrypted databases using the same password string/salt/iteration parameters as used in another platform’s SDK (at least, specifically, databases created with the Java SDK). I tried the following implementation of PBKDF2-SHA256 to load the raw bytes into the key and it seemed to match up with the raw bytes the Java implementation was calculating: https://gist.github.com/peteroupc/6986999

That’s interesting. If you use the raw key data instead of the password when creating the SymmetricKey, this should get around the issue. I was wondering why I could not open it using the sqlcipher command line. Right now we are not guaranteeing interoperability of databases that are encrypted because we don’t have an API to properly load them.

I don’t think there’s any API missing; it’s just that making encrypted databases compatible doesn’t seem generally useful.

The main use for making database files compatible between platforms is so that a db can be bundled in an app. But bundling an encrypted database in an app is sort of pointless because you then have to embed the key in the app too, negating the value of the encryption.

Correct, I used the output of that PBKDF snippet referenced as a raw key and it worked as expected.

In general I would tend to agree. However, our use case right now – valid or not – involves syncing a large number of documents that could be multiple megabytes in size each. While our primary client is using a marriage of Node and the Java SDK, it seems to be about a factor of five slower than its .NET counterpart. Too, we seem to be running out of heap space even when the JVM’s memory is increased to its (32-bit) maximum. Therefore, the idea is to use a small .NET utility to pre-load the database (as of the time of install) so the Java client can be started with minimal syncing. The key is randomly generated upon install and stored in the user’s vault/keychain à la the NPM package keytar (the Windows vault being of limited value but I digress), so the key does not have to be hard coded.

While our primary client is using a marriage of Node and the Java SDK, it seems to be about a factor of five slower than its .NET counterpart

Going through the REST API is a lot slower than making native calls (assuming your .NET client uses the native C# API.)

Actually, I’m confused — it sounds like you’re using Java and .NET in the same client? What platform is this on? And why do you need both? Running three virtual machines this way (including the JS one) sounds very inefficient.

Correct, the .NET client uses the native API.

No, they would be separate clients: there is a .NET client and a Node client. The intended .NET client is a bootstrap/utility program just to initialize the encrypted database. After that, the Node-based client process (the main application) uses JNI to initialize a Java VM and then utilizes the Java “native” CBLite SDK.

For what it’s worth I agree that it is terribly inefficient (thus the 32-bit/memory issue), but at this point the application architecture is an internal development issue that we cannot quickly resolve.

Not really looking for a technical solution there, though – just reporting the difference in the encryption algorithms. Thanks!

Not really looking for a technical solution there, though

Understood! Still, I’m concerned to hear about the performance + memory issues using Java + Node. I don’t know if you’ve done any analysis of this, but if you have any insights into what aspects are slow or memory-hungry, we’d love to hear about them.

While I’ve been able to attach jdb to the running embedded instance (and have it halt when it runs out of memory, list threads, etc), I haven’t yet sat down to get much farther than that.

In the mean time, I think I have the bootstrap utility at a good spot and was able to successfully create an encrypted database from the .NET CBL SDK and have the Java CBL SDK successfully read it.

If it helps anyone in the meantime, I was able to get CryptSharp from NuGet and use the CryptSharp.Utility.Pbkdf2.ComputeDerivedKey function parameterized with the HMACSHA256 hmacAlgorithm.