PHP script using lots of memory when bulk loading

[ingenthr]

Continuing the discussion from Couchbase use too much memory when insert data:

@ita1102

Today, I tried to import data into couchbase using PHP. But I think It’s use too much Memory( extremely larger than Data Size). When I insert 1GB data, it take more than 20GB RAM. This only occurs when data is inserting, after that, RAM usage reduce.
When I tried this PHP code, but insert into MySQL not Couchbase, it dont use as much as memory like that.
I used Ubuntu 14.04.

My initial guess would be that perhaps there’s something about how the script is reading the data. Is it small enough that you may post it here?

Also, what version of the PHP client? What version of libcouchbase underneath it?

[avsej]

@ita1102 if your data stored in JSON format and your parser requires the steam to be read into a string, then it likely explains why it need that much memory. But you have said with mysql it consumes less memory, does it mean, that the loader logic remains the same?