Best approach for high load application

#1

Hi guys, can anyone help me to choose best approach for next scenario of message processing:
So, i code a custom protocol for my IM in order to study network programming and highly loaded applications and i decided to choose NoSQL for store and process history messages and some other big things.
As i understood for every packet (my message packet) i need to create new document and insert my data.
It’s correct way to create one document per message or i can use something like in relational sqls ID with autoincrement?

Example, like:

Document_1
UserID = "1"
Text = “Test1”

Document_2
UserID = "1"
Text = “Test2”

Document_3
UserID = "2"
Text = “Test1”

Thanks for amazing database and client SDK.

#2

@Victor_Chicu

Regarding your best practice, it’s really going to depend on what you’re doing with the data and concurrency.

For example, it might be more efficient to have one document per conversation rather than one document per message. However, if you’re trying to add multiple messages simultaneously to a conversation document you’d have to deal with locks/concurrency to make sure one write doesn’t overwrite the other.

One document per conversation would have big advantages if you’re normally reading the data as a whole conversation. Trying to piece together a conversation from a bunch of individual message documents would be pretty inefficient.

Document size is also a factor, I generally try to keep the JSON document to a few KB at most.

Regarding your question about incrementing numbers, you can use the “Increment” function to provide this facility. Basically, you keep a single document that is just an incrementing number that tells you which document number to use next for the main document. You could also use GUIDs as another option. They add some overhead to reads and storage size because they’re larger, but remove some overhead on write because you don’t have to make an Increment call followed by an Insert call. Just depends on your use case.