I tried the same with the older Java client, 2.4.2. Unfortunately it is still showing inconsistent result. I wonder if it has to do with the cover index with select 1 as part of the query.
I had since slightly adjusted the query to
SELECT * FROM zs USE INDEX (zst USING GSI) WHERE (SELECT RAW 1 FROM (SELECT metadata.deleted AS deleted FROM zs AS ls USE INDEX (zst USING GSI) WHERE tag.token = $token AND tag.expiryepoch > $published ORDER BY metadata.published DESC LIMIT 1) AS lo WHERE deleted = FALSE) = 1 AND tag.token = $token AND outline.createresource = TRUE LIMIT 1
And it started 'working'. It consistently returned the wrong result for the first time, but subsequent query returns the correct result.
@vsr1 I will check the log once I gain access to the box.