Java REST put Attachment works for first Attachment, then returns error-Code 400

Hi,

i wrote a function to put Attachments into couchbase. The function seemed to work fine, but now it started to get error-code 400 from couchbase on the line where i get the InputStream with httpcon.getInputStream();

here is the function:

	/**
	 * 
	 * @param id - id of the document
	 * @param rev - last rev of the document
	 * @param filename 
	 * @param content 
	 * @param content_type - as "application/pdf"
	 * @param syncBucket - name of the bucket
	 * @param syncGatewayIP - ip without port
	 * @param username
	 * @param password
	 * @return new revision
	 */
	
	public static String putAttachmentByREST(String id, String rev, String filename, byte[] content, String content_type,
			String syncBucket, String syncGatewayIP, String username, String password) {
		String response = "failed";
		if (filename.contains(" ")) {
			filename = filename.replaceAll(" ", "");
		}
		try {
			URL url = new URL("http://" + syncGatewayIP + ":4984/" + syncBucket + "/" + id + "/"
					+ URLEncoder.encode(filename, "UTF-8") + "?rev=" + rev);
			log.info("putAttachment url: " + url.toString());
			HttpURLConnection httpCon = (HttpURLConnection) url.openConnection();
			httpCon.setDoOutput(true);
			httpCon.setDoInput(true);
			if (username != null && password != null && !username.isEmpty() && !password.isEmpty()) {
				setAuthInfo(httpCon, username, password);
			}
			httpCon.setRequestMethod("PUT");
			httpCon.setRequestProperty("Content-Type", content_type);
			OutputStream out = httpCon.getOutputStream();
			out.write(content);
			out.close();
			InputStream inStream = httpCon.getInputStream();
			BufferedReader br = new BufferedReader(new InputStreamReader(inStream));
			String line;
			StringBuilder result = new StringBuilder();
			while ((line = br.readLine()) != null) {
				result.append(line + "\n");
			}
			response = result.toString();
			JsonObject responseObject = JsonObject.fromJson(response);
			if (responseObject.containsKey("_rev")) {
				return responseObject.getString("_rev");
			} else if (responseObject.containsKey("rev")) {
				return responseObject.getString("rev");
			} else {
				return "";
			}
		} catch (MalformedURLException e) {
			log.error(e);
		} catch (ProtocolException e) {
			log.error(e);
		} catch (IOException e) {
			log.error(e);
		}
		return response;
	}

The Exception:

java.io.IOException: Server returned HTTP response code: 400 for URL: http://xx.xxx.xxx.x:4984/service_sync/thisisadocumentname/thisisanattachmentnamewhichisthesameasthedocumentname?rev=8-5393d79d9c552236de249795ca2966d8
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1840)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441)

I tried to get the error-log on TRACE-level but my application creates way too much output on that level.

Here is the calling code:

byte[] decoded = Base64.getDecoder().decode(attachment.getData());
					if(jsObject.containsKey("data")){
						jsObject.removeKey("data");
					}
					DatabaseManager.initialize();
					String rev = CouchbaseUtil.createOrUpdateCBDocumentByREST(id, jsObject, DatabaseManager.SYNC_USER);
					if(rev != null && !rev.isEmpty()){
						CouchbaseUtil.putAttachmentByREST(id, rev, id, decoded, attachment.getContent_type(), DatabaseManager.SYNC_USER);
					}

The function getRev gets the last revision of a document.

Its able to upload the first document of a whole series of pdfs but then starts throwing the IOException.
Also I am able to upload Attachments from plain text without a problem. I want to mention that the filename equals the ID of the document in the above example.

So my question is, why does it return 400? The function seems to be fine. Could this have to do with the format of the document?

Thanks in advance
Malte

The most efficient way to create a document with attachments is to PUT it as a multipart body. That way there’s only a single revision. What you’re doing creates one revision for the initial document, then another revision for each attachment.

Can you read the body of the response that has the 400 status? It will consist of JSON that describes the error.

Sorry for taking that long to answer on that thread. I was not able to read the response, but finally got it working, still producing 2 Revisions per update.

I would like to know how an Upload of an Attachment, together with its Document, would look like.
The Documentation clearly states, that you need the current revision, and the url for putting the attachment contains the Document-ID and the Attachment-Name. How can this be done in a multipart request? Could you please clear this up?

Right now, when i put an Attachment into Couchbase (from Server over the Sync-Gateway) I first create a Document, retrieve the revision, and then put the Attachment to the Document.
When i am updating a document with attachment, i first update the new content of the JsonDocument, which results in a new revision without Attachment, and then i put the new Attachment to the Document again after retrieving the current revision.

This process even seems to create Conflicts on these Documents, as there are multiple Documents that are read-only on the users application, but still contain conflicts.

I can not find any advice on the multipart upload, everybody just says to do it like that, but not how :slight_smile:
So please help us :wink:

You should read about MIME Multipart format.
Here’s a simple example:

Content-Type: multipart/mixed; boundary="BOUNDARY"

--BOUNDARY
Content-Type: application/json

{
	"_id": "THX-1138",
	"_rev": "1-foobar",
	"_attachments": {
		"mary.txt": {"type": "text/doggerel", "length": 52, "follows": true}
	}
}
--BOUNDARY  
Content-Type: text/doggerel
Content-Disposition: attachment; filename=mary.txt

Mary had a little lamb
Its fleece was white as snow
--BOUNDARY--

Great I will look into that, thank you very much for your quick reply.

How would the URL to put this to look like?
Just like the standard
http://ip.ip.ip.ip:port/bucketname/doc_id
?

Yes, it’s just like a regular document PUT. The Content-Type header value tells the server to expect multipart content.

Alright so i tried this with different contents and contenttypes and urls and with id and without id and with PUT and POST and so on…

Nothing seems to work. When i used the pdf and set the Content-Type to “application/pdf”, i got an IOException “Error writing to Server” (the PDF was about 0.5MB small), so no communcation seemed to work at all. When changing to text as an attachment with Content-Type “text/plain”, the server returned a ResponseCode of 415 as “Unsupported Media Type”. I remember putting Text in CB as Attachment in the beginning of me working with Couchbase, so i don’t think the “text/plain” is the problem. Also the 0.5MB PDF-Attachment shouldn’t be a problem for the server (our usual Attachments are between 1 and 15MB in size). So my initial thought is, that the requests ContentType with “multipart/mixed” creates the problems, even though I think that, if that was the problem, the approach with the PDF should have “worked” too.
I am trying to put the Text or PDF as a bytearray.

To create the multipart-Request, i changed an example I found here:
http://www.codejava.net/java-se/networking/upload-files-by-sending-multipart-request-programmatically

Before i use it, my built JsonObject looks like this:

{
 "name": "testObject",
 "_id": "testid",
 "_rev": "1-foobar",
 "_attachments": {
    "atta": {
       "length": 556826,
       "follows": true,
       "type": "application/pdf"
     }
  }
}

My URL: "http://xxx.xxx.xxx.xxx:4984/sync_bucket_name/testid"

I also tried this with and without _rev/_id
Basically what it does:
Constructor of the request:

    LINE_FEED = "\r\n";
    charset = "UTF-8";
    boundary = "BOUNDARY";  // just changed this for your example
    URL url = new URL(requestURL);
    httpConn = (HttpURLConnection) url.openConnection();
    httpConn.setUseCaches(false);
    httpConn.setDoOutput(true); 
    httpConn.setDoInput(true);
    httpConn.setRequestMethod(requestMethod);
    httpConn.setRequestProperty("Content-Type",
            "multipart/mixed; boundary=" + boundary);

    outputStream = httpConn.getOutputStream();
    writer = new PrintWriter(new OutputStreamWriter(outputStream, charset), true);

then the jsonPart is added:

public void addJsonPart(String jsonContent) throws IOException{
	writer.append("--" + boundary).append(LINE_FEED);
    writer.append("Content-Type: application/json")
            .append(LINE_FEED);
    writer.flush();

    outputStream.write(jsonContent.getBytes());
    outputStream.flush();
     
    writer.append(LINE_FEED);
    writer.flush();
}

then the Attachment is added, for this example i used the pdf-content

request.addByteArrayPart("atta", "application/pdf", getContent()); // getContent returns byte[], also tried it with contenttype: "application/octet-stream" in json and here

The Method addByteArrayPart:

 public void addByteArrayPart(String fileName, String contentType, byte[] content) throws IOException{
	writer.append("--" + boundary).append(LINE_FEED);
	writer.append(
            "Content-Type: " + contentType)
            .append(LINE_FEED);
    writer.append(
            "Content-Disposition: attachment; filename=" + fileName)
            .append(LINE_FEED);
    
    writer.append("Content-Transfer-Encoding: binary").append(LINE_FEED); // tried with and without this line
    writer.append(LINE_FEED);
    writer.flush();

    outputStream.write(content);
    outputStream.flush();
     
    writer.append(LINE_FEED);
    writer.flush();
}

As soon as the Request is executed by getting the ResponseCode or getting the InputStream of the Connection, the IOException will occur. Any Advice?

If I have put together a working example, I would make a Post on the forum, so that others can save themselves a lot of time (and you :wink: )

Our Sync-Gateway-Version is 1.3.1

If you need any further Information from me, let me know. BTW the standard put with a simple JsonObject works just fine. Next Step for me will be to test with an Apache API to make Multipart-Requests.

And the Server/SyncGateway logs

 415 Invalid content type multipart/mixed

For Content-Type multipart/related it returns Status 500
and for multipart/form-data the 415 again.