You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by Chad Woodhead <ch...@gmail.com> on 2019/02/13 18:03:01 UTC
Failed to read TOC File
I use the org.apache.nifi.provenance.WriteAheadProvenanceRepository and I
am seeing the following error in my logs a lot and I can't view any
provenance data in the UI:
2019-02-13 12:57:44,637 ERROR [Compress Provenance Logs-1-thread-1]
o.a.n.p.s.EventFileCompressor Failed to read TOC File
/data/disk5/nifi/provenance_repository/toc/158994812.toc
java.io.EOFException: null
at
org.apache.nifi.provenance.toc.StandardTocReader.<init>(StandardTocReader.java:48)
at
org.apache.nifi.provenance.serialization.EventFileCompressor.run(EventFileCompressor.java:93)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Any ideas on what could be going on?
-Chad
Re: Failed to read TOC File
Posted by Chad Woodhead <ch...@gmail.com>.
Thanks Mark. I really appreciate the help. I'll take a look at these in my
different clusters.
-Chad
On Wed, Feb 13, 2019 at 3:09 PM Mark Payne <ma...@hotmail.com> wrote:
> Depending on the size of the nodes, you may want to also increase the
> number of indexing threads
> ("nifi.provenance.repository.index.threads"). You may want to also
> increase the amount of time to store provenance
> from 24 hours to something like 36 months... for most people just limiting
> based on size is enough. The time is really
> only useful if you are worried about it from a compliance standpoint, such
> that you are only allowed to store that sort
> of data for a limited amount of time. (Maybe you can only store PII for 24
> hours, for instance, and PII is included in your
> attributes).
>
> The value for "nifi.bored.yield.duration" can probably be scaled back from
> "10 millis" to something like "1 millis". This comes
> into play when a processor's incoming queue is empty or outgoing queue is
> full, for instance. It will wait approximately 10 miliseconds
> before checking if the issue has resolved. This results in lower CPU
> usage, but it also leads to "artificial latency." For a production
> server, "1 millis" is probably just fine.
>
> You may also want to consider changing the values of
> "nifi.content.repository.archive.max.retention.period" and
> "nifi.content.repository.archive.max.usage.percentage"
> When you store the provenance data, it's often helpful to be able to view
> the data as it was at that point in the flow. These properties control how
> long you keep around this data after you've finished processing it, so
> that you can go back and look at it for debugging purposes, etc.
>
> These are probably the most critical things, in terms of performance and
> utilization.
>
> Thanks
> -Mark
>
>
> On Feb 13, 2019, at 2:31 PM, Chad Woodhead <ch...@gmail.com> wrote:
>
> Mark,
>
> That must be it! I have "nifi.provenance.repository.max.storage.size" = 1
> GB. Just bumped that to 200 GB like you suggested and I can see provenance
> again. I've always wondered why my provenance partitions never got very
> large!
>
> While we're on the subject, are there other settings like this that based
> on their default values I could be under-utilizing the resources (storage,
> mem, CPU, etc.) I have on my servers dedicated to NiFi?
>
> -Chad
>
> On Wed, Feb 13, 2019 at 1:48 PM Mark Payne <ma...@hotmail.com> wrote:
>
>> Hey Chad,
>>
>> What do you have for the value of the
>> "nifi.provenance.repository.max.storage.size" property?
>> We will often see this if the value is very small (the default is 1 GB,
>> which is very small) and the volume
>> of data is reasonably high.
>>
>> The way that the repo works, it writes to one file for a while, until
>> that file reaches 100 MB or up to 30 seconds,
>> by default (configured via "nifi.provenance.repository.rollover.size" and
>> "nifi.provenance.repository.rollover.time").
>> At that point, it rolls over to writing to a new file and adds the
>> now-completed file to a queue. A background thread
>> is then responsible for compressing that completed file.
>>
>> What can happen, though, if the max storage space is small is that the
>> data can actually be aged off from the repository
>> before that background task attempts to compress it. That can result in
>> either a FileNotFoundException or an EOFException
>> when trying to read the TOC file (depending on the timing of when the
>> age-off happens). It could potentially occur on the
>> .prov file, in addition to, or instead of the .toc file.
>>
>> So generally, the solution is to increase the max storage size. It looks
>> like you have 130 GB on each partition and 2 partitions per
>> node, so 260 GB total per node that you can use for provenance. So I
>> would set the max storage size to something like "200 GB".
>> Since it is a soft limit and it may use more disk space than that
>> temporarily before shrinking back down, you'll want to give it a little
>> bit of wiggle room.
>>
>> Thanks
>> -Mark
>>
>>
>> On Feb 13, 2019, at 1:31 PM, Chad Woodhead <ch...@gmail.com>
>> wrote:
>>
>> Hey Joe,
>>
>> Yes nifi.provenance.repository.implementation=
>> org.apache.nifi.provenance.WriteAheadProvenanceRepository
>>
>> Disk space is fine as well. I have dedicated mounts for provenance (as
>> well all the repos have their own dedicated mounts):
>>
>> nifi.provenance.repository.dir.default=
>> /data/disk5/nifi/provenance_repository
>> nifi.provenance.repository.directory.provenance2=
>> /data/disk6/nifi/provenance_repository
>>
>> Both of these mounts have plenty of space and are only 1% full and have
>> never become close to being filled up.
>>
>> <image.png>
>>
>> -Chad
>>
>> On Wed, Feb 13, 2019 at 1:06 PM Joe Witt <jo...@gmail.com> wrote:
>>
>>> Chad,
>>>
>>> In your conf/nifi.properties please see what the implementation is for
>>> your provenance repository. This specied on
>>>
>>>
>>> nifi.provenance.repository.implementation=org.apache.nifi.provenance.WriteAheadProvenanceRepository
>>>
>>> Is that what you have?
>>>
>>> The above error I believe could occur if the location where provenance
>>> is being written runs out of disk space. It is important to ensure that
>>> prov is sized and on a partition alone where this wont happen. This is
>>> also true for the flow file repo. The content repo is more resilient to
>>> this by design but still you want all three repo areas on their own
>>> partitions as per best practices.
>>>
>>> Thanks
>>> Joe
>>>
>>> On Wed, Feb 13, 2019 at 1:03 PM Chad Woodhead <ch...@gmail.com>
>>> wrote:
>>>
>>>> I use the org.apache.nifi.provenance.WriteAheadProvenanceRepository and
>>>> I am seeing the following error in my logs a lot and I can't view any
>>>> provenance data in the UI:
>>>>
>>>> 2019-02-13 12:57:44,637 ERROR [Compress Provenance Logs-1-thread-1]
>>>> o.a.n.p.s.EventFileCompressor Failed to read TOC File
>>>> /data/disk5/nifi/provenance_repository/toc/158994812.toc
>>>> java.io.EOFException: null
>>>> at
>>>> org.apache.nifi.provenance.toc.StandardTocReader.<init>(StandardTocReader.java:48)
>>>> at
>>>> org.apache.nifi.provenance.serialization.EventFileCompressor.run(EventFileCompressor.java:93)
>>>> at
>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>>> at
>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>>> at
>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>>> at java.lang.Thread.run(Thread.java:745)
>>>>
>>>> Any ideas on what could be going on?
>>>>
>>>> -Chad
>>>>
>>>
>>
>
Re: Failed to read TOC File
Posted by Mark Payne <ma...@hotmail.com>.
Depending on the size of the nodes, you may want to also increase the number of indexing threads
("nifi.provenance.repository.index.threads"). You may want to also increase the amount of time to store provenance
from 24 hours to something like 36 months... for most people just limiting based on size is enough. The time is really
only useful if you are worried about it from a compliance standpoint, such that you are only allowed to store that sort
of data for a limited amount of time. (Maybe you can only store PII for 24 hours, for instance, and PII is included in your
attributes).
The value for "nifi.bored.yield.duration" can probably be scaled back from "10 millis" to something like "1 millis". This comes
into play when a processor's incoming queue is empty or outgoing queue is full, for instance. It will wait approximately 10 miliseconds
before checking if the issue has resolved. This results in lower CPU usage, but it also leads to "artificial latency." For a production
server, "1 millis" is probably just fine.
You may also want to consider changing the values of "nifi.content.repository.archive.max.retention.period" and "nifi.content.repository.archive.max.usage.percentage"
When you store the provenance data, it's often helpful to be able to view the data as it was at that point in the flow. These properties control how
long you keep around this data after you've finished processing it, so that you can go back and look at it for debugging purposes, etc.
These are probably the most critical things, in terms of performance and utilization.
Thanks
-Mark
On Feb 13, 2019, at 2:31 PM, Chad Woodhead <ch...@gmail.com>> wrote:
Mark,
That must be it! I have "nifi.provenance.repository.max.storage.size" = 1 GB. Just bumped that to 200 GB like you suggested and I can see provenance again. I've always wondered why my provenance partitions never got very large!
While we're on the subject, are there other settings like this that based on their default values I could be under-utilizing the resources (storage, mem, CPU, etc.) I have on my servers dedicated to NiFi?
-Chad
On Wed, Feb 13, 2019 at 1:48 PM Mark Payne <ma...@hotmail.com>> wrote:
Hey Chad,
What do you have for the value of the "nifi.provenance.repository.max.storage.size" property?
We will often see this if the value is very small (the default is 1 GB, which is very small) and the volume
of data is reasonably high.
The way that the repo works, it writes to one file for a while, until that file reaches 100 MB or up to 30 seconds,
by default (configured via "nifi.provenance.repository.rollover.size" and "nifi.provenance.repository.rollover.time").
At that point, it rolls over to writing to a new file and adds the now-completed file to a queue. A background thread
is then responsible for compressing that completed file.
What can happen, though, if the max storage space is small is that the data can actually be aged off from the repository
before that background task attempts to compress it. That can result in either a FileNotFoundException or an EOFException
when trying to read the TOC file (depending on the timing of when the age-off happens). It could potentially occur on the
.prov file, in addition to, or instead of the .toc file.
So generally, the solution is to increase the max storage size. It looks like you have 130 GB on each partition and 2 partitions per
node, so 260 GB total per node that you can use for provenance. So I would set the max storage size to something like "200 GB".
Since it is a soft limit and it may use more disk space than that temporarily before shrinking back down, you'll want to give it a little
bit of wiggle room.
Thanks
-Mark
On Feb 13, 2019, at 1:31 PM, Chad Woodhead <ch...@gmail.com>> wrote:
Hey Joe,
Yes nifi.provenance.repository.implementation=org.apache.nifi.provenance.WriteAheadProvenanceRepository
Disk space is fine as well. I have dedicated mounts for provenance (as well all the repos have their own dedicated mounts):
nifi.provenance.repository.dir.default=/data/disk5/nifi/provenance_repository
nifi.provenance.repository.directory.provenance2=/data/disk6/nifi/provenance_repository
Both of these mounts have plenty of space and are only 1% full and have never become close to being filled up.
<image.png>
-Chad
On Wed, Feb 13, 2019 at 1:06 PM Joe Witt <jo...@gmail.com>> wrote:
Chad,
In your conf/nifi.properties please see what the implementation is for your provenance repository. This specied on
nifi.provenance.repository.implementation=org.apache.nifi.provenance.WriteAheadProvenanceRepository
Is that what you have?
The above error I believe could occur if the location where provenance is being written runs out of disk space. It is important to ensure that prov is sized and on a partition alone where this wont happen. This is also true for the flow file repo. The content repo is more resilient to this by design but still you want all three repo areas on their own partitions as per best practices.
Thanks
Joe
On Wed, Feb 13, 2019 at 1:03 PM Chad Woodhead <ch...@gmail.com>> wrote:
I use the org.apache.nifi.provenance.WriteAheadProvenanceRepository and I am seeing the following error in my logs a lot and I can't view any provenance data in the UI:
2019-02-13 12:57:44,637 ERROR [Compress Provenance Logs-1-thread-1] o.a.n.p.s.EventFileCompressor Failed to read TOC File /data/disk5/nifi/provenance_repository/toc/158994812.toc
java.io.EOFException: null
at org.apache.nifi.provenance.toc.StandardTocReader.<init>(StandardTocReader.java:48)
at org.apache.nifi.provenance.serialization.EventFileCompressor.run(EventFileCompressor.java:93)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Any ideas on what could be going on?
-Chad
Re: Failed to read TOC File
Posted by Chad Woodhead <ch...@gmail.com>.
Mark,
That must be it! I have "nifi.provenance.repository.max.storage.size" = 1
GB. Just bumped that to 200 GB like you suggested and I can see provenance
again. I've always wondered why my provenance partitions never got very
large!
While we're on the subject, are there other settings like this that based
on their default values I could be under-utilizing the resources (storage,
mem, CPU, etc.) I have on my servers dedicated to NiFi?
-Chad
On Wed, Feb 13, 2019 at 1:48 PM Mark Payne <ma...@hotmail.com> wrote:
> Hey Chad,
>
> What do you have for the value of the
> "nifi.provenance.repository.max.storage.size" property?
> We will often see this if the value is very small (the default is 1 GB,
> which is very small) and the volume
> of data is reasonably high.
>
> The way that the repo works, it writes to one file for a while, until that
> file reaches 100 MB or up to 30 seconds,
> by default (configured via "nifi.provenance.repository.rollover.size" and
> "nifi.provenance.repository.rollover.time").
> At that point, it rolls over to writing to a new file and adds the
> now-completed file to a queue. A background thread
> is then responsible for compressing that completed file.
>
> What can happen, though, if the max storage space is small is that the
> data can actually be aged off from the repository
> before that background task attempts to compress it. That can result in
> either a FileNotFoundException or an EOFException
> when trying to read the TOC file (depending on the timing of when the
> age-off happens). It could potentially occur on the
> .prov file, in addition to, or instead of the .toc file.
>
> So generally, the solution is to increase the max storage size. It looks
> like you have 130 GB on each partition and 2 partitions per
> node, so 260 GB total per node that you can use for provenance. So I would
> set the max storage size to something like "200 GB".
> Since it is a soft limit and it may use more disk space than that
> temporarily before shrinking back down, you'll want to give it a little
> bit of wiggle room.
>
> Thanks
> -Mark
>
>
> On Feb 13, 2019, at 1:31 PM, Chad Woodhead <ch...@gmail.com> wrote:
>
> Hey Joe,
>
> Yes nifi.provenance.repository.implementation=
> org.apache.nifi.provenance.WriteAheadProvenanceRepository
>
> Disk space is fine as well. I have dedicated mounts for provenance (as
> well all the repos have their own dedicated mounts):
>
> nifi.provenance.repository.dir.default=
> /data/disk5/nifi/provenance_repository
> nifi.provenance.repository.directory.provenance2=
> /data/disk6/nifi/provenance_repository
>
> Both of these mounts have plenty of space and are only 1% full and have
> never become close to being filled up.
>
> <image.png>
>
> -Chad
>
> On Wed, Feb 13, 2019 at 1:06 PM Joe Witt <jo...@gmail.com> wrote:
>
>> Chad,
>>
>> In your conf/nifi.properties please see what the implementation is for
>> your provenance repository. This specied on
>>
>>
>> nifi.provenance.repository.implementation=org.apache.nifi.provenance.WriteAheadProvenanceRepository
>>
>> Is that what you have?
>>
>> The above error I believe could occur if the location where provenance is
>> being written runs out of disk space. It is important to ensure that prov
>> is sized and on a partition alone where this wont happen. This is also
>> true for the flow file repo. The content repo is more resilient to this by
>> design but still you want all three repo areas on their own partitions as
>> per best practices.
>>
>> Thanks
>> Joe
>>
>> On Wed, Feb 13, 2019 at 1:03 PM Chad Woodhead <ch...@gmail.com>
>> wrote:
>>
>>> I use the org.apache.nifi.provenance.WriteAheadProvenanceRepository and
>>> I am seeing the following error in my logs a lot and I can't view any
>>> provenance data in the UI:
>>>
>>> 2019-02-13 12:57:44,637 ERROR [Compress Provenance Logs-1-thread-1]
>>> o.a.n.p.s.EventFileCompressor Failed to read TOC File
>>> /data/disk5/nifi/provenance_repository/toc/158994812.toc
>>> java.io.EOFException: null
>>> at
>>> org.apache.nifi.provenance.toc.StandardTocReader.<init>(StandardTocReader.java:48)
>>> at
>>> org.apache.nifi.provenance.serialization.EventFileCompressor.run(EventFileCompressor.java:93)
>>> at
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>> at java.lang.Thread.run(Thread.java:745)
>>>
>>> Any ideas on what could be going on?
>>>
>>> -Chad
>>>
>>
>
Re: Failed to read TOC File
Posted by Mark Payne <ma...@hotmail.com>.
Hey Chad,
What do you have for the value of the "nifi.provenance.repository.max.storage.size" property?
We will often see this if the value is very small (the default is 1 GB, which is very small) and the volume
of data is reasonably high.
The way that the repo works, it writes to one file for a while, until that file reaches 100 MB or up to 30 seconds,
by default (configured via "nifi.provenance.repository.rollover.size" and "nifi.provenance.repository.rollover.time").
At that point, it rolls over to writing to a new file and adds the now-completed file to a queue. A background thread
is then responsible for compressing that completed file.
What can happen, though, if the max storage space is small is that the data can actually be aged off from the repository
before that background task attempts to compress it. That can result in either a FileNotFoundException or an EOFException
when trying to read the TOC file (depending on the timing of when the age-off happens). It could potentially occur on the
.prov file, in addition to, or instead of the .toc file.
So generally, the solution is to increase the max storage size. It looks like you have 130 GB on each partition and 2 partitions per
node, so 260 GB total per node that you can use for provenance. So I would set the max storage size to something like "200 GB".
Since it is a soft limit and it may use more disk space than that temporarily before shrinking back down, you'll want to give it a little
bit of wiggle room.
Thanks
-Mark
On Feb 13, 2019, at 1:31 PM, Chad Woodhead <ch...@gmail.com>> wrote:
Hey Joe,
Yes nifi.provenance.repository.implementation=org.apache.nifi.provenance.WriteAheadProvenanceRepository
Disk space is fine as well. I have dedicated mounts for provenance (as well all the repos have their own dedicated mounts):
nifi.provenance.repository.dir.default=/data/disk5/nifi/provenance_repository
nifi.provenance.repository.directory.provenance2=/data/disk6/nifi/provenance_repository
Both of these mounts have plenty of space and are only 1% full and have never become close to being filled up.
<image.png>
-Chad
On Wed, Feb 13, 2019 at 1:06 PM Joe Witt <jo...@gmail.com>> wrote:
Chad,
In your conf/nifi.properties please see what the implementation is for your provenance repository. This specied on
nifi.provenance.repository.implementation=org.apache.nifi.provenance.WriteAheadProvenanceRepository
Is that what you have?
The above error I believe could occur if the location where provenance is being written runs out of disk space. It is important to ensure that prov is sized and on a partition alone where this wont happen. This is also true for the flow file repo. The content repo is more resilient to this by design but still you want all three repo areas on their own partitions as per best practices.
Thanks
Joe
On Wed, Feb 13, 2019 at 1:03 PM Chad Woodhead <ch...@gmail.com>> wrote:
I use the org.apache.nifi.provenance.WriteAheadProvenanceRepository and I am seeing the following error in my logs a lot and I can't view any provenance data in the UI:
2019-02-13 12:57:44,637 ERROR [Compress Provenance Logs-1-thread-1] o.a.n.p.s.EventFileCompressor Failed to read TOC File /data/disk5/nifi/provenance_repository/toc/158994812.toc
java.io.EOFException: null
at org.apache.nifi.provenance.toc.StandardTocReader.<init>(StandardTocReader.java:48)
at org.apache.nifi.provenance.serialization.EventFileCompressor.run(EventFileCompressor.java:93)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Any ideas on what could be going on?
-Chad
Re: Failed to read TOC File
Posted by Chad Woodhead <ch...@gmail.com>.
Hey Joe,
Yes nifi.provenance.repository.implementation=
org.apache.nifi.provenance.WriteAheadProvenanceRepository
Disk space is fine as well. I have dedicated mounts for provenance (as well
all the repos have their own dedicated mounts):
nifi.provenance.repository.dir.default=
/data/disk5/nifi/provenance_repository
nifi.provenance.repository.directory.provenance2=
/data/disk6/nifi/provenance_repository
Both of these mounts have plenty of space and are only 1% full and have
never become close to being filled up.
[image: image.png]
-Chad
On Wed, Feb 13, 2019 at 1:06 PM Joe Witt <jo...@gmail.com> wrote:
> Chad,
>
> In your conf/nifi.properties please see what the implementation is for
> your provenance repository. This specied on
>
>
> nifi.provenance.repository.implementation=org.apache.nifi.provenance.WriteAheadProvenanceRepository
>
> Is that what you have?
>
> The above error I believe could occur if the location where provenance is
> being written runs out of disk space. It is important to ensure that prov
> is sized and on a partition alone where this wont happen. This is also
> true for the flow file repo. The content repo is more resilient to this by
> design but still you want all three repo areas on their own partitions as
> per best practices.
>
> Thanks
> Joe
>
> On Wed, Feb 13, 2019 at 1:03 PM Chad Woodhead <ch...@gmail.com>
> wrote:
>
>> I use the org.apache.nifi.provenance.WriteAheadProvenanceRepository and I
>> am seeing the following error in my logs a lot and I can't view any
>> provenance data in the UI:
>>
>> 2019-02-13 12:57:44,637 ERROR [Compress Provenance Logs-1-thread-1]
>> o.a.n.p.s.EventFileCompressor Failed to read TOC File
>> /data/disk5/nifi/provenance_repository/toc/158994812.toc
>> java.io.EOFException: null
>> at
>> org.apache.nifi.provenance.toc.StandardTocReader.<init>(StandardTocReader.java:48)
>> at
>> org.apache.nifi.provenance.serialization.EventFileCompressor.run(EventFileCompressor.java:93)
>> at
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:745)
>>
>> Any ideas on what could be going on?
>>
>> -Chad
>>
>
Re: Failed to read TOC File
Posted by Joe Witt <jo...@gmail.com>.
Chad,
In your conf/nifi.properties please see what the implementation is for your
provenance repository. This specied on
nifi.provenance.repository.implementation=org.apache.nifi.provenance.WriteAheadProvenanceRepository
Is that what you have?
The above error I believe could occur if the location where provenance is
being written runs out of disk space. It is important to ensure that prov
is sized and on a partition alone where this wont happen. This is also
true for the flow file repo. The content repo is more resilient to this by
design but still you want all three repo areas on their own partitions as
per best practices.
Thanks
Joe
On Wed, Feb 13, 2019 at 1:03 PM Chad Woodhead <ch...@gmail.com>
wrote:
> I use the org.apache.nifi.provenance.WriteAheadProvenanceRepository and I
> am seeing the following error in my logs a lot and I can't view any
> provenance data in the UI:
>
> 2019-02-13 12:57:44,637 ERROR [Compress Provenance Logs-1-thread-1]
> o.a.n.p.s.EventFileCompressor Failed to read TOC File
> /data/disk5/nifi/provenance_repository/toc/158994812.toc
> java.io.EOFException: null
> at
> org.apache.nifi.provenance.toc.StandardTocReader.<init>(StandardTocReader.java:48)
> at
> org.apache.nifi.provenance.serialization.EventFileCompressor.run(EventFileCompressor.java:93)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
>
> Any ideas on what could be going on?
>
> -Chad
>