You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@couchdb.apache.org by GitBox <gi...@apache.org> on 2020/01/24 19:13:22 UTC
[GitHub] [couchdb] hamedminaee opened a new issue #2485: Error:
504 Gateway Time-out
\nThe server didn't respond in
time.\n\n
hamedminaee opened a new issue #2485: Error: <html><body><h1>504 Gateway Time-out</h1>\nThe server didn't respond in time.\n</body></html>\n
URL: https://github.com/apache/couchdb/issues/2485
## Description
Hi couch team,
we use couchdb v3/preview deployed in ocp and recently we noticed a very strange behavior.
so when we have data greater than 4k in couch and we send a get query to get the indexed result as follows:
` const searchResults = await db.search("searchAll", "searchAll", {
include_docs: true,
limit: limit >= 0 ? limit : 10,
bookmark,
counts,
query: query && query.length > 0 ? query : "*:*",
sort,
});`
we get the following error from couchdb:
`Error: <html><body><h1>504 Gateway Time-out</h1>\nThe server didn't respond in time.\n</body></html>\n`
It is noteworthy that this only happens when we make this call for the very first time on data greater that 4k but if we make this call again everything works fine.
## Expected Behaviour
We should get the search result
## Your Environment
{"couchdb":"Welcome","version":"3.0.0-8ac108a","git_sha":"8ac108a","uuid":"47db6f0128f090e2bc32658f5db310a5","features":["search","partitioned","pluggable-storage-engines","reshard","scheduler"],"vendor":{"name":"The Apache Software Foundation"}}
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [couchdb] kocolosk commented on issue #2485: Error:
504 Gateway Time-out
\nThe server didn't respond in
time.\n\n
Posted by GitBox <gi...@apache.org>.
kocolosk commented on issue #2485: Error: <html><body><h1>504 Gateway Time-out</h1>\nThe server didn't respond in time.\n</body></html>\n
URL: https://github.com/apache/couchdb/issues/2485#issuecomment-579325741
@wohali it looks like @hamedminaee is running a build of 8ac108a, which should include the automatic index warming. I don't see any bug fixes in master after that commit.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [couchdb] willholley commented on issue #2485: Error:
504 Gateway Time-out
\nThe server didn't respond in
time.\n\n
Posted by GitBox <gi...@apache.org>.
willholley commented on issue #2485: Error: <html><body><h1>504 Gateway Time-out</h1>\nThe server didn't respond in time.\n</body></html>\n
URL: https://github.com/apache/couchdb/issues/2485#issuecomment-578697538
@hamedminaee it sounds like your index is just taking some time to build. When you query a search index, the request will block until the index has completed building. If the index build does not complete within the timeout, it will fail with an error. Then on the second request the index has likely completed building and so succeeds.
Are you able to confirm this by checking that the index has built prior to the first query? You can check using the [search info](https://docs.couchdb.org/en/latest/http-api.html#cap-/{db}/_design/{ddoc}/_search_info/{index}) endpoint. You could also check the logs for any timeout-related errors.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [couchdb] wohali commented on issue #2485: Error:
504 Gateway Time-out
\nThe server didn't respond in
time.\n\n
Posted by GitBox <gi...@apache.org>.
wohali commented on issue #2485: Error: <html><body><h1>504 Gateway Time-out</h1>\nThe server didn't respond in time.\n</body></html>\n
URL: https://github.com/apache/couchdb/issues/2485#issuecomment-578936785
@hamedminaee Will is not correct about indexes being warmed in the background in your version of CouchDB; this is only true with Cloudant and CouchDB 3.0.0+.
You must access your index the first time if you want to build an index after creating it, in CouchDB 2.x.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [couchdb] wohali commented on issue #2485: Error:
504 Gateway Time-out
\nThe server didn't respond in
time.\n\n
Posted by GitBox <gi...@apache.org>.
wohali commented on issue #2485: Error: <html><body><h1>504 Gateway Time-out</h1>\nThe server didn't respond in time.\n</body></html>\n
URL: https://github.com/apache/couchdb/issues/2485#issuecomment-598968173
Sounds like this has been resolved.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [couchdb] willholley commented on issue #2485: Error:
504 Gateway Time-out
\nThe server didn't respond in
time.\n\n
Posted by GitBox <gi...@apache.org>.
willholley commented on issue #2485: Error: <html><body><h1>504 Gateway Time-out</h1>\nThe server didn't respond in time.\n</body></html>\n
URL: https://github.com/apache/couchdb/issues/2485#issuecomment-578838137
I'd suggest a simpler index and/or use Query (`_find` endpoint) instead for the query (indexing performance is better as it doesn't need to marshall data to JavaScript).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [couchdb] willholley commented on issue #2485: Error:
504 Gateway Time-out
\nThe server didn't respond in
time.\n\n
Posted by GitBox <gi...@apache.org>.
willholley commented on issue #2485: Error: <html><body><h1>504 Gateway Time-out</h1>\nThe server didn't respond in time.\n</body></html>\n
URL: https://github.com/apache/couchdb/issues/2485#issuecomment-578768269
@hamedminaee indexes should be warmed in the background, though that process may not happen quickly enough for you if you're inserting a document and then immediately querying for it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [couchdb] hamedminaee commented on issue #2485: Error:
504 Gateway Time-out
\nThe server didn't respond in
time.\n\n
Posted by GitBox <gi...@apache.org>.
hamedminaee commented on issue #2485: Error: <html><body><h1>504 Gateway Time-out</h1>\nThe server didn't respond in time.\n</body></html>\n
URL: https://github.com/apache/couchdb/issues/2485#issuecomment-578836379
@willholley so in that case if we need to quickly query the result after insertion what is the best way to handle it(if there is a way)?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [couchdb] wohali closed issue #2485: Error: 504
Gateway Time-out
\nThe server didn't respond in time.\n\n
Posted by GitBox <gi...@apache.org>.
wohali closed issue #2485: Error: <html><body><h1>504 Gateway Time-out</h1>\nThe server didn't respond in time.\n</body></html>\n
URL: https://github.com/apache/couchdb/issues/2485
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [couchdb] hamedminaee commented on issue #2485: Error:
504 Gateway Time-out
\nThe server didn't respond in
time.\n\n
Posted by GitBox <gi...@apache.org>.
hamedminaee commented on issue #2485: Error: <html><body><h1>504 Gateway Time-out</h1>\nThe server didn't respond in time.\n</body></html>\n
URL: https://github.com/apache/couchdb/issues/2485#issuecomment-578745886
@willholley Thanks will. That is right before the first call there is no index. So if indexing is taking longer than the timeout what is the best way to handle it?
Currently in my code I added a check and as soon the timeout error happens I do another call to recover.
What do you think?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [couchdb] willholley commented on issue #2485: Error:
504 Gateway Time-out
\nThe server didn't respond in
time.\n\n
Posted by GitBox <gi...@apache.org>.
willholley commented on issue #2485: Error: <html><body><h1>504 Gateway Time-out</h1>\nThe server didn't respond in time.\n</body></html>\n
URL: https://github.com/apache/couchdb/issues/2485#issuecomment-578750778
@hamedminaee at what point are you creating your index? The expectation is that index creation is a rare operation and would typically be handled outside of the querying code.
In terms of how you handle your index not being up to date, that's somewhat dependent on your application. If it makes sense, you could retry the query a number of times. Another option is to use the `stale` flag which will bypass the check that the index is up to date on the server, if you want a response quickly and don't mind the results potentially being out of date.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [couchdb] hamedminaee commented on issue #2485: Error:
504 Gateway Time-out
\nThe server didn't respond in
time.\n\n
Posted by GitBox <gi...@apache.org>.
hamedminaee commented on issue #2485: Error: <html><body><h1>504 Gateway Time-out</h1>\nThe server didn't respond in time.\n</body></html>\n
URL: https://github.com/apache/couchdb/issues/2485#issuecomment-578764573
@willholley Sorry I think I misunderstood your question at the beginning
In terms of index creation we create the index as soon as we insert the document to the database not at query time.
```
public async create(
databaseName: String) {
const status = await this._db.create(databaseName)
if (status.ok) {
const db = this._db.use(databaseName)
const docs = await db.insert(DesignDocumentGenerator.searchAll)
if (docs.ok) {
this._logger.debug({
log: "Insert design document with STIX index",
});
return db
}
throw "Can't create index"
}
throw "Can't create database"
}
```
However if I get that correctly the indexing process starts as soon as we query that document for the first and that is where it takes a bit of time to build the indexes.
So if thats the case does that mean that we should make a dummy get query for the inserted doc at the insertion time to initiate the indexing process rather than relying on user to make a call?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services