The Java SDK provides some value-add over calling the query REST API directly, most notably that it will automatically create concurrent connections to the query service when needed (up to 12, by default), plus we have experience in using Java Netty in an efficient way.
You may be able to get some small efficiencies from calling the REST API directly, simply because you will be able to parse the results directly into your target form rather than going through an intermediate QueryResult.
But personally I would leave this very far down any list of potential optimisations. I would first focus on getting the query itself fast, including checking the indexes, and making sure my code was efficiently performing the queries in sufficient parallelism. I would also look at whether some of the queries can be performed using the Key-Value API instead - if oyu know the keys, it is usually faster to perform KV operations.