You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by "Filipe Manana (JIRA)" <ji...@apache.org> on 2011/08/30 20:57:38 UTC
[jira] [Created] (COUCHDB-1266) Add stats field to _active_tasks
Add stats field to _active_tasks
--------------------------------
Key: COUCHDB-1266
URL: https://issues.apache.org/jira/browse/COUCHDB-1266
Project: CouchDB
Issue Type: Improvement
Reporter: Filipe Manana
Assignee: Filipe Manana
Fix For: 1.2
This proposal is simply to add a "stats" field to the _active_tasks results. This field can be an arbitrary JSON value and each task can set it to whatever is appropriate for it.The following patch also defines some basic stats for the existing tasks:
1) database compaction - # changes done, total changes, # of revisions copied, # of attachments copied and progress (an integer percentage, same as what is exposed in the existing text field "status");
2) view compaction - # of ids copied, total number of ids, # of kvs copied, total number of kvs and progress
3) view indexing - # changes done, total changes, # inserted kvs, # deleted kvs, progress
4) replication - # missing revisions checked, # missing revisions found, # docs read, # docs written, # doc write failures, source seq number, checkpointed source seq number, progress.
A screenshot of Futon with 3 different tasks:
http://dl.dropbox.com/u/25067962/active_tasks_stats.png
Patch at:
https://github.com/fdmanana/couchdb/compare/task_stats.diff
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1266) Add stats field to _active_tasks
Posted by "Filipe Manana (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13106979#comment-13106979 ]
Filipe Manana commented on COUCHDB-1266:
----------------------------------------
I just rebased the patch on current trunk, after Paul's view refactoring:
https://github.com/fdmanana/couchdb/commit/9c5481157fc47090a20fe098eb18a7cdcc823231
I'll rename this ticket's title, as the purpose is no longer to add a "stats" field but rather to make _active_tasks more generic, less rigid, and allow different tasks to specify different properties.
> Add stats field to _active_tasks
> --------------------------------
>
> Key: COUCHDB-1266
> URL: https://issues.apache.org/jira/browse/COUCHDB-1266
> Project: CouchDB
> Issue Type: Improvement
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Fix For: 1.2
>
> Attachments: alternate-api.patch
>
>
> This proposal is simply to add a "stats" field to the _active_tasks results. This field can be an arbitrary JSON value and each task can set it to whatever is appropriate for it.The following patch also defines some basic stats for the existing tasks:
> 1) database compaction - # changes done, total changes, # of revisions copied, # of attachments copied and progress (an integer percentage, same as what is exposed in the existing text field "status");
> 2) view compaction - # of ids copied, total number of ids, # of kvs copied, total number of kvs and progress
> 3) view indexing - # changes done, total changes, # inserted kvs, # deleted kvs, progress
> 4) replication - # missing revisions checked, # missing revisions found, # docs read, # docs written, # doc write failures, source seq number, checkpointed source seq number, progress.
> A screenshot of Futon with 3 different tasks:
> http://dl.dropbox.com/u/25067962/active_tasks_stats.png
> Patch at:
> https://github.com/fdmanana/couchdb/compare/task_stats.diff
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1266) Add stats field to _active_tasks
Posted by "Paul Joseph Davis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13094081#comment-13094081 ]
Paul Joseph Davis commented on COUCHDB-1266:
--------------------------------------------
Awesome.
Two comments:
1. I don't see a reason to expose the JSON in Futon. The JSON is surely for script consumption. I was mulling over something similar to this and was thinking of replacing the current status entry with JSON like you have and then Futon and other tools would just build the status string client side. This would allow scripts to consume the same information without resorting to string processing.
Also, I always found it quite annoying to be writing user facing text deep in the various internals like view indexing or compaction.
2. I think it be a good idea for the stats to include the db name and view/design name when appropriate.
> Add stats field to _active_tasks
> --------------------------------
>
> Key: COUCHDB-1266
> URL: https://issues.apache.org/jira/browse/COUCHDB-1266
> Project: CouchDB
> Issue Type: Improvement
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Fix For: 1.2
>
>
> This proposal is simply to add a "stats" field to the _active_tasks results. This field can be an arbitrary JSON value and each task can set it to whatever is appropriate for it.The following patch also defines some basic stats for the existing tasks:
> 1) database compaction - # changes done, total changes, # of revisions copied, # of attachments copied and progress (an integer percentage, same as what is exposed in the existing text field "status");
> 2) view compaction - # of ids copied, total number of ids, # of kvs copied, total number of kvs and progress
> 3) view indexing - # changes done, total changes, # inserted kvs, # deleted kvs, progress
> 4) replication - # missing revisions checked, # missing revisions found, # docs read, # docs written, # doc write failures, source seq number, checkpointed source seq number, progress.
> A screenshot of Futon with 3 different tasks:
> http://dl.dropbox.com/u/25067962/active_tasks_stats.png
> Patch at:
> https://github.com/fdmanana/couchdb/compare/task_stats.diff
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (COUCHDB-1266) Make _active_tasks more flexible
Posted by "Filipe Manana (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Filipe Manana resolved COUCHDB-1266.
------------------------------------
Resolution: Fixed
> Make _active_tasks more flexible
> --------------------------------
>
> Key: COUCHDB-1266
> URL: https://issues.apache.org/jira/browse/COUCHDB-1266
> Project: CouchDB
> Issue Type: Improvement
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Fix For: 1.2
>
> Attachments: alternate-api.patch
>
>
> This proposal is simply to allow the output of _active_tasks to be less rigid. Basically to allow each task to be able to output different JSON fields.
> Somethings like the status text simply go away. Instead application can built it based on more granular fields provided by _active_tasks.
> Some examples:
> 1) "progress" (an integer percentage, for all tasks)
> 2) "database" (for compactions and indexer tasks)
> 3) "design_document" (for indexer and view compaction tasks)
> 4) "source" and "target" (for replications)
> 5) "docs_read", "docs_written", "doc_write_failures",
> "missing_revs_found", "missing_revs_checked", "source_seq",
> "checkpointed_source_seq" and "continuous" for replications
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (COUCHDB-1266) Add stats field to _active_tasks
Posted by "Paul Joseph Davis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Paul Joseph Davis updated COUCHDB-1266:
---------------------------------------
Attachment: alternate-api.patch
After thinking about it for a few more minutes there's no reason to even carry the prop list around, we can just give couch_task_status an API to mutate its internal task representation. This saves even more code and makes the whole thing quite a bit more friendly.
> Add stats field to _active_tasks
> --------------------------------
>
> Key: COUCHDB-1266
> URL: https://issues.apache.org/jira/browse/COUCHDB-1266
> Project: CouchDB
> Issue Type: Improvement
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Fix For: 1.2
>
> Attachments: alternate-api.patch
>
>
> This proposal is simply to add a "stats" field to the _active_tasks results. This field can be an arbitrary JSON value and each task can set it to whatever is appropriate for it.The following patch also defines some basic stats for the existing tasks:
> 1) database compaction - # changes done, total changes, # of revisions copied, # of attachments copied and progress (an integer percentage, same as what is exposed in the existing text field "status");
> 2) view compaction - # of ids copied, total number of ids, # of kvs copied, total number of kvs and progress
> 3) view indexing - # changes done, total changes, # inserted kvs, # deleted kvs, progress
> 4) replication - # missing revisions checked, # missing revisions found, # docs read, # docs written, # doc write failures, source seq number, checkpointed source seq number, progress.
> A screenshot of Futon with 3 different tasks:
> http://dl.dropbox.com/u/25067962/active_tasks_stats.png
> Patch at:
> https://github.com/fdmanana/couchdb/compare/task_stats.diff
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1266) Add stats field to _active_tasks
Posted by "Paul Joseph Davis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13094085#comment-13094085 ]
Paul Joseph Davis commented on COUCHDB-1266:
--------------------------------------------
In fact, the more I think about it, I don't see why we wouldn't just create a single JSON blob for each task, then all clients can do whatever it is they want to do with the data. I'd also get rid of things like progress and instead focus on allowing clients to do their own calculations and display.
> Add stats field to _active_tasks
> --------------------------------
>
> Key: COUCHDB-1266
> URL: https://issues.apache.org/jira/browse/COUCHDB-1266
> Project: CouchDB
> Issue Type: Improvement
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Fix For: 1.2
>
>
> This proposal is simply to add a "stats" field to the _active_tasks results. This field can be an arbitrary JSON value and each task can set it to whatever is appropriate for it.The following patch also defines some basic stats for the existing tasks:
> 1) database compaction - # changes done, total changes, # of revisions copied, # of attachments copied and progress (an integer percentage, same as what is exposed in the existing text field "status");
> 2) view compaction - # of ids copied, total number of ids, # of kvs copied, total number of kvs and progress
> 3) view indexing - # changes done, total changes, # inserted kvs, # deleted kvs, progress
> 4) replication - # missing revisions checked, # missing revisions found, # docs read, # docs written, # doc write failures, source seq number, checkpointed source seq number, progress.
> A screenshot of Futon with 3 different tasks:
> http://dl.dropbox.com/u/25067962/active_tasks_stats.png
> Patch at:
> https://github.com/fdmanana/couchdb/compare/task_stats.diff
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1266) Make _active_tasks more flexible
Posted by "Filipe Manana (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13107008#comment-13107008 ]
Filipe Manana commented on COUCHDB-1266:
----------------------------------------
Applied to trunk and branch 1.2.x.
Here's an _active_tasks example output:
$ curl http://localhost:5984/_active_tasks
[
{
"pid":"<0.242.0>",
"changes_done":31209,
"database":"indexer_test_3",
"design_document":"_design/test",
"progress":5,
"started_on":1316228432,
"total_changes":551201,
"type":"indexer",
"updated_on":1316228461
}
],
{
"pid":"<0.1156.0>",
"database":"indexer_test_3",
"design_document":"_design/test",
"progress":21,
"started_on":1316229336,
"type":"view_compaction",
"updated_on":1316229352
},
{
"pid":"<0.1303.0>",
"checkpointed_source_seq":17333,
"continuous":false,
"doc_write_failures":0,
"docs_read":17833,
"docs_written":17833,
"missing_revisions_found":17833,
"progress":3,
"revisions_checked":17833,
"source":"http://fdmanana.couchone.com/indexer_test/",
"source_seq":551202,
"started_on":1316229471,
"target":"indexer_test",
"type":"replication",
"updated_on":1316230082
}
]
> Make _active_tasks more flexible
> --------------------------------
>
> Key: COUCHDB-1266
> URL: https://issues.apache.org/jira/browse/COUCHDB-1266
> Project: CouchDB
> Issue Type: Improvement
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Fix For: 1.2
>
> Attachments: alternate-api.patch
>
>
> This proposal is simply to allow the output of _active_tasks to be less rigid. Basically to allow each task to be able to output different JSON fields.
> Somethings like the status text simply go away. Instead application can built it based on more granular fields provided by _active_tasks.
> Some examples:
> 1) "progress" (an integer percentage, for all tasks)
> 2) "database" (for compactions and indexer tasks)
> 3) "design_document" (for indexer and view compaction tasks)
> 4) "source" and "target" (for replications)
> 5) "docs_read", "docs_written", "doc_write_failures",
> "missing_revs_found", "missing_revs_checked", "source_seq",
> "checkpointed_source_seq" and "continuous" for replications
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1266) Add stats field to _active_tasks
Posted by "Paul Joseph Davis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13094124#comment-13094124 ]
Paul Joseph Davis commented on COUCHDB-1266:
--------------------------------------------
Ooh. If we added a couple proplist helpers like couch_util:set_value/3 and couch_util:incr_value/3 this could trim out a lot of redundant code.
> Add stats field to _active_tasks
> --------------------------------
>
> Key: COUCHDB-1266
> URL: https://issues.apache.org/jira/browse/COUCHDB-1266
> Project: CouchDB
> Issue Type: Improvement
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Fix For: 1.2
>
>
> This proposal is simply to add a "stats" field to the _active_tasks results. This field can be an arbitrary JSON value and each task can set it to whatever is appropriate for it.The following patch also defines some basic stats for the existing tasks:
> 1) database compaction - # changes done, total changes, # of revisions copied, # of attachments copied and progress (an integer percentage, same as what is exposed in the existing text field "status");
> 2) view compaction - # of ids copied, total number of ids, # of kvs copied, total number of kvs and progress
> 3) view indexing - # changes done, total changes, # inserted kvs, # deleted kvs, progress
> 4) replication - # missing revisions checked, # missing revisions found, # docs read, # docs written, # doc write failures, source seq number, checkpointed source seq number, progress.
> A screenshot of Futon with 3 different tasks:
> http://dl.dropbox.com/u/25067962/active_tasks_stats.png
> Patch at:
> https://github.com/fdmanana/couchdb/compare/task_stats.diff
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1266) Add stats field to _active_tasks
Posted by "Adam Kocoloski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13094117#comment-13094117 ]
Adam Kocoloski commented on COUCHDB-1266:
-----------------------------------------
Totally agree that it's silly for us to be formatting status messages in the internals. I like what you've done here, Filipe, but I would have also supported a patch where a task was specified as a simple property list.
> Add stats field to _active_tasks
> --------------------------------
>
> Key: COUCHDB-1266
> URL: https://issues.apache.org/jira/browse/COUCHDB-1266
> Project: CouchDB
> Issue Type: Improvement
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Fix For: 1.2
>
>
> This proposal is simply to add a "stats" field to the _active_tasks results. This field can be an arbitrary JSON value and each task can set it to whatever is appropriate for it.The following patch also defines some basic stats for the existing tasks:
> 1) database compaction - # changes done, total changes, # of revisions copied, # of attachments copied and progress (an integer percentage, same as what is exposed in the existing text field "status");
> 2) view compaction - # of ids copied, total number of ids, # of kvs copied, total number of kvs and progress
> 3) view indexing - # changes done, total changes, # inserted kvs, # deleted kvs, progress
> 4) replication - # missing revisions checked, # missing revisions found, # docs read, # docs written, # doc write failures, source seq number, checkpointed source seq number, progress.
> A screenshot of Futon with 3 different tasks:
> http://dl.dropbox.com/u/25067962/active_tasks_stats.png
> Patch at:
> https://github.com/fdmanana/couchdb/compare/task_stats.diff
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1266) Add stats field to _active_tasks
Posted by "Paul Joseph Davis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13094200#comment-13094200 ]
Paul Joseph Davis commented on COUCHDB-1266:
--------------------------------------------
Ah, I see what you mean now. Yeah, I'd probably just improve the task tab so that it has a table for each type of task and then each table can have the columns and formatting it needs.
> Add stats field to _active_tasks
> --------------------------------
>
> Key: COUCHDB-1266
> URL: https://issues.apache.org/jira/browse/COUCHDB-1266
> Project: CouchDB
> Issue Type: Improvement
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Fix For: 1.2
>
> Attachments: alternate-api.patch
>
>
> This proposal is simply to add a "stats" field to the _active_tasks results. This field can be an arbitrary JSON value and each task can set it to whatever is appropriate for it.The following patch also defines some basic stats for the existing tasks:
> 1) database compaction - # changes done, total changes, # of revisions copied, # of attachments copied and progress (an integer percentage, same as what is exposed in the existing text field "status");
> 2) view compaction - # of ids copied, total number of ids, # of kvs copied, total number of kvs and progress
> 3) view indexing - # changes done, total changes, # inserted kvs, # deleted kvs, progress
> 4) replication - # missing revisions checked, # missing revisions found, # docs read, # docs written, # doc write failures, source seq number, checkpointed source seq number, progress.
> A screenshot of Futon with 3 different tasks:
> http://dl.dropbox.com/u/25067962/active_tasks_stats.png
> Patch at:
> https://github.com/fdmanana/couchdb/compare/task_stats.diff
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1266) Add stats field to _active_tasks
Posted by "Filipe Manana (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13094143#comment-13094143 ]
Filipe Manana commented on COUCHDB-1266:
----------------------------------------
Thanks Paul and Adam,
I agree the current status text field could go away, as it can be built from this stats property. However I would be more confortable doing this change for 2.0 for e.g.
I like to see this in Futon. I agree it currently doesn't look like very formatted and so on, so it could get polished by someone more confortable with Futon/CSS/HTML.
About the progress, right now it's trivial to compute (and it's more of an estimation, without a very high degree of accuracy). But i prefer to leave it's computation to the tasks and not the applications. This is because how it's calculated can change in the future as the implementation of the tasks (compaction, indexing, replication) evolve.
I agree that with setelement and #record.fieldname tricks some code size can be reduced.
> Add stats field to _active_tasks
> --------------------------------
>
> Key: COUCHDB-1266
> URL: https://issues.apache.org/jira/browse/COUCHDB-1266
> Project: CouchDB
> Issue Type: Improvement
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Fix For: 1.2
>
>
> This proposal is simply to add a "stats" field to the _active_tasks results. This field can be an arbitrary JSON value and each task can set it to whatever is appropriate for it.The following patch also defines some basic stats for the existing tasks:
> 1) database compaction - # changes done, total changes, # of revisions copied, # of attachments copied and progress (an integer percentage, same as what is exposed in the existing text field "status");
> 2) view compaction - # of ids copied, total number of ids, # of kvs copied, total number of kvs and progress
> 3) view indexing - # changes done, total changes, # inserted kvs, # deleted kvs, progress
> 4) replication - # missing revisions checked, # missing revisions found, # docs read, # docs written, # doc write failures, source seq number, checkpointed source seq number, progress.
> A screenshot of Futon with 3 different tasks:
> http://dl.dropbox.com/u/25067962/active_tasks_stats.png
> Patch at:
> https://github.com/fdmanana/couchdb/compare/task_stats.diff
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1266) Add stats field to _active_tasks
Posted by "Filipe Manana (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13096969#comment-13096969 ]
Filipe Manana commented on COUCHDB-1266:
----------------------------------------
Patch updated,
https://github.com/fdmanana/couchdb/compare/task_stats.diff
The status text string in futon is now constructed by JavaScript via the better structured _active_tasks output.
> Add stats field to _active_tasks
> --------------------------------
>
> Key: COUCHDB-1266
> URL: https://issues.apache.org/jira/browse/COUCHDB-1266
> Project: CouchDB
> Issue Type: Improvement
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Fix For: 1.2
>
> Attachments: alternate-api.patch
>
>
> This proposal is simply to add a "stats" field to the _active_tasks results. This field can be an arbitrary JSON value and each task can set it to whatever is appropriate for it.The following patch also defines some basic stats for the existing tasks:
> 1) database compaction - # changes done, total changes, # of revisions copied, # of attachments copied and progress (an integer percentage, same as what is exposed in the existing text field "status");
> 2) view compaction - # of ids copied, total number of ids, # of kvs copied, total number of kvs and progress
> 3) view indexing - # changes done, total changes, # inserted kvs, # deleted kvs, progress
> 4) replication - # missing revisions checked, # missing revisions found, # docs read, # docs written, # doc write failures, source seq number, checkpointed source seq number, progress.
> A screenshot of Futon with 3 different tasks:
> http://dl.dropbox.com/u/25067962/active_tasks_stats.png
> Patch at:
> https://github.com/fdmanana/couchdb/compare/task_stats.diff
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (COUCHDB-1266) Make _active_tasks more flexible
Posted by "Filipe Manana (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Filipe Manana updated COUCHDB-1266:
-----------------------------------
Description:
This proposal is simply to allow the output of _active_tasks to be less rigid. Basically to allow each task to be able to output different JSON fields.
Somethings like the status text simply go away. Instead application can built it based on more granular fields provided by _active_tasks.
Some examples:
1) "progress" (an integer percentage, for all tasks)
2) "database" (for compactions and indexer tasks)
3) "design_document" (for indexer and view compaction tasks)
4) "source" and "target" (for replications)
5) "docs_read", "docs_written", "doc_write_failures",
"missing_revs_found", "missing_revs_checked", "source_seq",
"checkpointed_source_seq" and "continuous" for replications
was:
This proposal is simply to add a "stats" field to the _active_tasks results. This field can be an arbitrary JSON value and each task can set it to whatever is appropriate for it.The following patch also defines some basic stats for the existing tasks:
1) database compaction - # changes done, total changes, # of revisions copied, # of attachments copied and progress (an integer percentage, same as what is exposed in the existing text field "status");
2) view compaction - # of ids copied, total number of ids, # of kvs copied, total number of kvs and progress
3) view indexing - # changes done, total changes, # inserted kvs, # deleted kvs, progress
4) replication - # missing revisions checked, # missing revisions found, # docs read, # docs written, # doc write failures, source seq number, checkpointed source seq number, progress.
A screenshot of Futon with 3 different tasks:
http://dl.dropbox.com/u/25067962/active_tasks_stats.png
Patch at:
https://github.com/fdmanana/couchdb/compare/task_stats.diff
Summary: Make _active_tasks more flexible (was: Add stats field to _active_tasks)
> Make _active_tasks more flexible
> --------------------------------
>
> Key: COUCHDB-1266
> URL: https://issues.apache.org/jira/browse/COUCHDB-1266
> Project: CouchDB
> Issue Type: Improvement
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Fix For: 1.2
>
> Attachments: alternate-api.patch
>
>
> This proposal is simply to allow the output of _active_tasks to be less rigid. Basically to allow each task to be able to output different JSON fields.
> Somethings like the status text simply go away. Instead application can built it based on more granular fields provided by _active_tasks.
> Some examples:
> 1) "progress" (an integer percentage, for all tasks)
> 2) "database" (for compactions and indexer tasks)
> 3) "design_document" (for indexer and view compaction tasks)
> 4) "source" and "target" (for replications)
> 5) "docs_read", "docs_written", "doc_write_failures",
> "missing_revs_found", "missing_revs_checked", "source_seq",
> "checkpointed_source_seq" and "continuous" for replications
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (COUCHDB-1266) Add stats field to _active_tasks
Posted by "Paul Joseph Davis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Paul Joseph Davis updated COUCHDB-1266:
---------------------------------------
Attachment: alternate-api.diff
Not sure why we'd want to delay changing the status code until 2.0. 1.2 seems like a big enough version that we could improve this part of the code.
Why would you want to see this stuff in Futon? If Futon is doing its job all of the relevant information should be represented in the UI. I can't imagine that normal people are going to be interested in seeing a JSON blob duplicating all of the status information that's already formatted for display.
I guess its possible that the definition of progress changes in the future. Not sure I see how it would change enough to be considerably different yet the same enough that it's interpreted the same.
As to the record comment I'm not sure where that's coming from. The suggestion on using a proplist is so that we can carry the task information around and update it more gracefully. As an example I did a quick sketch of what this might look like for the view compactor. Attached is a diff from trunk showing what I was thinking.
> Add stats field to _active_tasks
> --------------------------------
>
> Key: COUCHDB-1266
> URL: https://issues.apache.org/jira/browse/COUCHDB-1266
> Project: CouchDB
> Issue Type: Improvement
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Fix For: 1.2
>
> Attachments: alternate-api.diff
>
>
> This proposal is simply to add a "stats" field to the _active_tasks results. This field can be an arbitrary JSON value and each task can set it to whatever is appropriate for it.The following patch also defines some basic stats for the existing tasks:
> 1) database compaction - # changes done, total changes, # of revisions copied, # of attachments copied and progress (an integer percentage, same as what is exposed in the existing text field "status");
> 2) view compaction - # of ids copied, total number of ids, # of kvs copied, total number of kvs and progress
> 3) view indexing - # changes done, total changes, # inserted kvs, # deleted kvs, progress
> 4) replication - # missing revisions checked, # missing revisions found, # docs read, # docs written, # doc write failures, source seq number, checkpointed source seq number, progress.
> A screenshot of Futon with 3 different tasks:
> http://dl.dropbox.com/u/25067962/active_tasks_stats.png
> Patch at:
> https://github.com/fdmanana/couchdb/compare/task_stats.diff
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1266) Add stats field to _active_tasks
Posted by "Paul Joseph Davis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13106980#comment-13106980 ]
Paul Joseph Davis commented on COUCHDB-1266:
--------------------------------------------
Awesome work, Filipe. +1 on committing this as it is.
> Add stats field to _active_tasks
> --------------------------------
>
> Key: COUCHDB-1266
> URL: https://issues.apache.org/jira/browse/COUCHDB-1266
> Project: CouchDB
> Issue Type: Improvement
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Fix For: 1.2
>
> Attachments: alternate-api.patch
>
>
> This proposal is simply to add a "stats" field to the _active_tasks results. This field can be an arbitrary JSON value and each task can set it to whatever is appropriate for it.The following patch also defines some basic stats for the existing tasks:
> 1) database compaction - # changes done, total changes, # of revisions copied, # of attachments copied and progress (an integer percentage, same as what is exposed in the existing text field "status");
> 2) view compaction - # of ids copied, total number of ids, # of kvs copied, total number of kvs and progress
> 3) view indexing - # changes done, total changes, # inserted kvs, # deleted kvs, progress
> 4) replication - # missing revisions checked, # missing revisions found, # docs read, # docs written, # doc write failures, source seq number, checkpointed source seq number, progress.
> A screenshot of Futon with 3 different tasks:
> http://dl.dropbox.com/u/25067962/active_tasks_stats.png
> Patch at:
> https://github.com/fdmanana/couchdb/compare/task_stats.diff
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1266) Add stats field to _active_tasks
Posted by "Filipe Manana (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13094203#comment-13094203 ]
Filipe Manana commented on COUCHDB-1266:
----------------------------------------
Exactly Paul.
I'll be changing the "type" field to be easier to handle with, for example for view compaction it is the string "View Group Compaction" - something like "view_compaction" would be easier for applications. Then Futon, using a simple switch/if-then-else logic can display "View Group Compaction" or something else.
> Add stats field to _active_tasks
> --------------------------------
>
> Key: COUCHDB-1266
> URL: https://issues.apache.org/jira/browse/COUCHDB-1266
> Project: CouchDB
> Issue Type: Improvement
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Fix For: 1.2
>
> Attachments: alternate-api.patch
>
>
> This proposal is simply to add a "stats" field to the _active_tasks results. This field can be an arbitrary JSON value and each task can set it to whatever is appropriate for it.The following patch also defines some basic stats for the existing tasks:
> 1) database compaction - # changes done, total changes, # of revisions copied, # of attachments copied and progress (an integer percentage, same as what is exposed in the existing text field "status");
> 2) view compaction - # of ids copied, total number of ids, # of kvs copied, total number of kvs and progress
> 3) view indexing - # changes done, total changes, # inserted kvs, # deleted kvs, progress
> 4) replication - # missing revisions checked, # missing revisions found, # docs read, # docs written, # doc write failures, source seq number, checkpointed source seq number, progress.
> A screenshot of Futon with 3 different tasks:
> http://dl.dropbox.com/u/25067962/active_tasks_stats.png
> Patch at:
> https://github.com/fdmanana/couchdb/compare/task_stats.diff
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (COUCHDB-1266) Add stats field to _active_tasks
Posted by "Paul Joseph Davis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Paul Joseph Davis updated COUCHDB-1266:
---------------------------------------
Attachment: (was: alternate-api.diff)
> Add stats field to _active_tasks
> --------------------------------
>
> Key: COUCHDB-1266
> URL: https://issues.apache.org/jira/browse/COUCHDB-1266
> Project: CouchDB
> Issue Type: Improvement
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Fix For: 1.2
>
>
> This proposal is simply to add a "stats" field to the _active_tasks results. This field can be an arbitrary JSON value and each task can set it to whatever is appropriate for it.The following patch also defines some basic stats for the existing tasks:
> 1) database compaction - # changes done, total changes, # of revisions copied, # of attachments copied and progress (an integer percentage, same as what is exposed in the existing text field "status");
> 2) view compaction - # of ids copied, total number of ids, # of kvs copied, total number of kvs and progress
> 3) view indexing - # changes done, total changes, # inserted kvs, # deleted kvs, progress
> 4) replication - # missing revisions checked, # missing revisions found, # docs read, # docs written, # doc write failures, source seq number, checkpointed source seq number, progress.
> A screenshot of Futon with 3 different tasks:
> http://dl.dropbox.com/u/25067962/active_tasks_stats.png
> Patch at:
> https://github.com/fdmanana/couchdb/compare/task_stats.diff
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1266) Add stats field to _active_tasks
Posted by "Filipe Manana (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13094194#comment-13094194 ]
Filipe Manana commented on COUCHDB-1266:
----------------------------------------
I see what you mean now Paul.
I still think the progress field should be given by couch and is one of the fields people care most about.
As for Futon, specially for replication tasks, I see all (or most) of those stats meaningful for the normal user. Other than displaying the json object, I don't see a way to display these fields which are not common for all tasks, I guess someone specialized on UIs would find a much better way.
> Add stats field to _active_tasks
> --------------------------------
>
> Key: COUCHDB-1266
> URL: https://issues.apache.org/jira/browse/COUCHDB-1266
> Project: CouchDB
> Issue Type: Improvement
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Fix For: 1.2
>
> Attachments: alternate-api.patch
>
>
> This proposal is simply to add a "stats" field to the _active_tasks results. This field can be an arbitrary JSON value and each task can set it to whatever is appropriate for it.The following patch also defines some basic stats for the existing tasks:
> 1) database compaction - # changes done, total changes, # of revisions copied, # of attachments copied and progress (an integer percentage, same as what is exposed in the existing text field "status");
> 2) view compaction - # of ids copied, total number of ids, # of kvs copied, total number of kvs and progress
> 3) view indexing - # changes done, total changes, # inserted kvs, # deleted kvs, progress
> 4) replication - # missing revisions checked, # missing revisions found, # docs read, # docs written, # doc write failures, source seq number, checkpointed source seq number, progress.
> A screenshot of Futon with 3 different tasks:
> http://dl.dropbox.com/u/25067962/active_tasks_stats.png
> Patch at:
> https://github.com/fdmanana/couchdb/compare/task_stats.diff
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira