admin and web dashboard

[z11373 <z1...@outlook.com>]

Hi,
I have few questions below regarding to Accumulo:

1. I login to accumulo shell as root, then I create a table and insert a row
with visibility = "foobar". Unless I grant "foobar" authz to user root, I am
not able to read that row. My question is in case I don't grant that authz
and a month later I forgot which string I assigned to column visibility, how
can I (as user root) to see all rows regardless the user has the proper
authz or not? It's like super admin user who has access to all rows
regardless the column visibility being assigned to those rows. Of course if
I know the authz then I can grant that to user, but in this case I forgot or
don't know
 
2. We can insert same key with different value, and by default it'll show
the latest version (given the max version setting is set to 1), if I never
delete that key-value, will Accumulo delete all previous versions?
 
3. Does Accumulo have a web based dashboard? Right now I can just run the
shell to see the tables and rows belong to those tables, but it'd be nice if
there is a web dashboard that can be used to view Accumulo tables.

Thanks,
Z



--
View this message in context: http://apache-accumulo.1065345.n5.nabble.com/admin-and-web-dashboard-tp14347.html
Sent from the Users mailing list archive at Nabble.com.

Re: admin and web dashboard

[z11373 <z1...@outlook.com>]

Awesome! It works! I now can see the .rf file in HDFS. Thanks Josh and Mike!



--
View this message in context: http://apache-accumulo.1065345.n5.nabble.com/admin-and-web-dashboard-tp14347p14354.html
Sent from the Users mailing list archive at Nabble.com.

Re: admin and web dashboard

[Mike Drob <ma...@cloudera.com>]

Also might need to run a 'flush -t $table -w'

On Mon, Jun 8, 2015 at 1:39 PM, Josh Elser <jo...@gmail.com> wrote:

> Since 1.5, All of Accumulo files are stored in HDFS: RFiles and WALs.
>
> Tables have the name you provide, but also maintain an internal unique ID
> to make operations like renaming easy. You can see this mapping via `tables
> -l` in the Accumulo shell.
>
> Given the ID for a table, you should be able to find all rfiles for a
> table "/accumulo/tables/$id/**/*.rf". If you don't see any rfiles there,
> run a `compact -t $table -w` and then check HDFS again.
>
>
> z11373 wrote:
>
>> That makes sense, thanks Josh!
>> Btw, where can I find the .rf files? I looked at under Accumulo install
>> folder and also /tmp, and couldn't find them. I also look at hdfs, and
>> only
>> found the folder, i.e. /accumulo/tables/n/default_tablet (where 'n' is a
>> number), and no files under that hdfs dir. I want to try the command
>> 'accumulo rfile-info' you mentioned earlier.
>>
>> Thanks again,
>> zainal
>>
>>
>>
>> --
>> View this message in context:
>> http://apache-accumulo.1065345.n5.nabble.com/admin-and-web-dashboard-tp14347p14351.html
>> Sent from the Users mailing list archive at Nabble.com.
>>
>

Re: admin and web dashboard

[Josh Elser <jo...@gmail.com>]

Since 1.5, All of Accumulo files are stored in HDFS: RFiles and WALs.

Tables have the name you provide, but also maintain an internal unique 
ID to make operations like renaming easy. You can see this mapping via 
`tables -l` in the Accumulo shell.

Given the ID for a table, you should be able to find all rfiles for a 
table "/accumulo/tables/$id/**/*.rf". If you don't see any rfiles there, 
run a `compact -t $table -w` and then check HDFS again.

z11373 wrote:
> That makes sense, thanks Josh!
> Btw, where can I find the .rf files? I looked at under Accumulo install
> folder and also /tmp, and couldn't find them. I also look at hdfs, and only
> found the folder, i.e. /accumulo/tables/n/default_tablet (where 'n' is a
> number), and no files under that hdfs dir. I want to try the command
> 'accumulo rfile-info' you mentioned earlier.
>
> Thanks again,
> zainal
>
>
>
> --
> View this message in context: http://apache-accumulo.1065345.n5.nabble.com/admin-and-web-dashboard-tp14347p14351.html
> Sent from the Users mailing list archive at Nabble.com.

Re: admin and web dashboard

[z11373 <z1...@outlook.com>]

That makes sense, thanks Josh!
Btw, where can I find the .rf files? I looked at under Accumulo install
folder and also /tmp, and couldn't find them. I also look at hdfs, and only
found the folder, i.e. /accumulo/tables/n/default_tablet (where 'n' is a
number), and no files under that hdfs dir. I want to try the command
'accumulo rfile-info' you mentioned earlier.

Thanks again,
zainal



--
View this message in context: http://apache-accumulo.1065345.n5.nabble.com/admin-and-web-dashboard-tp14347p14351.html
Sent from the Users mailing list archive at Nabble.com.

Re: admin and web dashboard

[Josh Elser <jo...@gmail.com>]

Right, you got it.

You can change the maxVersions many times but that won't be persisted on 
disk until a compaction rewrites that data. Scans will, however, always 
be consistent with the current configuration.

In the implementation, Accumulo will instantiate the current iterators 
for the minc or majc scope (depending on if it's a minor or major 
compaction) and write the output of memory/files being compacted, 
filtered/transformed by those iterators, back out to a single file.

z11373 wrote:
> Thanks Josh for the answers. I just have one clarification from the answer
> for #2.
> So let say we set the max version to keep is default value (in this case is
> 1), then after few inserts with same key, and we change the max version to
> 3. Based on what you said, if the compaction occurs before we change to 3,
> then we'll only see 1 row from scan (since all previous versions have been
> deleted), is that correct?
> If this is confirmed, then it looks like Accumulo is looking for that max
> version to keep value while doing compaction, so that it knows how many of
> previous versions need to be deleted, right?
>
> Thanks,
> Z
>
>
>
> --
> View this message in context: http://apache-accumulo.1065345.n5.nabble.com/admin-and-web-dashboard-tp14347p14349.html
> Sent from the Users mailing list archive at Nabble.com.

Re: admin and web dashboard

[z11373 <z1...@outlook.com>]

Thanks Josh for the answers. I just have one clarification from the answer
for #2.
So let say we set the max version to keep is default value (in this case is
1), then after few inserts with same key, and we change the max version to
3. Based on what you said, if the compaction occurs before we change to 3,
then we'll only see 1 row from scan (since all previous versions have been
deleted), is that correct?
If this is confirmed, then it looks like Accumulo is looking for that max
version to keep value while doing compaction, so that it knows how many of
previous versions need to be deleted, right?

Thanks,
Z



--
View this message in context: http://apache-accumulo.1065345.n5.nabble.com/admin-and-web-dashboard-tp14347p14349.html
Sent from the Users mailing list archive at Nabble.com.

Re: admin and web dashboard

[Josh Elser <jo...@gmail.com>]

Hi -- answers inline. Hope they help.

z11373 wrote:
> Hi,
> I have few questions below regarding to Accumulo:
>
> 1. I login to accumulo shell as root, then I create a table and insert a row
> with visibility = "foobar". Unless I grant "foobar" authz to user root, I am
> not able to read that row. My question is in case I don't grant that authz
> and a month later I forgot which string I assigned to column visibility, how
> can I (as user root) to see all rows regardless the user has the proper
> authz or not? It's like super admin user who has access to all rows
> regardless the column visibility being assigned to those rows. Of course if
> I know the authz then I can grant that to user, but in this case I forgot or
> don't know

Allowing a user to bypass visibility labels is against one of the core 
principals behind the design of Accumulo, and, as such, is not allowed 
or supported. The only way to inspect data w/o being subject to Accumulo 
security is to inspect the files in HDFS (e.g. `accumulo rfile-info 
/path/to/rfile.rf`)

> 2. We can insert same key with different value, and by default it'll show
> the latest version (given the max version setting is set to 1), if I never
> delete that key-value, will Accumulo delete all previous versions?

By default when you create a table, a VersioningIterator is attached to 
that table which limits the persisted versions for a Key (matching 
row+cf+cq+cv) to one. Accumulo doesn't proactively deleted the older 
version when you write a new one, but only filters until a compaction 
occurs. When the files are re-written in the compaction, all but the 
newest version would be "deleted" (by omitting them from the new file).

You can change the number of versions kept with the VersioningIterator 
by the maxVersions option, or you can remove it completely from the 
table and retain all versions.

> 3. Does Accumulo have a web based dashboard? Right now I can just run the
> shell to see the tables and rows belong to those tables, but it'd be nice if
> there is a web dashboard that can be used to view Accumulo tables.

The Accumulo Monitor[1] is a web application that provides an 
administrative overview of an Accumulo system. It includes metrics like 
query and ingest rate, error message aggregation, and per-table statistics.

By default, there is no means to view the contents of a table via the 
Monitor. If you enable the monitor to run with SSL[2], you will have the 
option to launch an Accumulo shell in your web browser. This is done to 
ensure that prying eyes cannot see the data in transit as it travels 
from the Monitor to your browser.

[1] http://accumulo.apache.org/screenshots.html
[2] http://accumulo.apache.org/1.7/accumulo_user_manual.html#_ssl_2

> Thanks,
> Z
>
>
>
> --
> View this message in context: http://apache-accumulo.1065345.n5.nabble.com/admin-and-web-dashboard-tp14347.html
> Sent from the Users mailing list archive at Nabble.com.