You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@trafficserver.apache.org by Patrick McManus <mc...@ducksong.com> on 2010/08/11 18:36:37 UTC
ts 2.0.0 failed assert `FATAL: CacheDir.cc:693
Hi all -
I'm working on a trafficserver plugin - and if I spin my ~40 case test
suite through TS repeatedly I can pretty easily reproduce the assert
below. (It is not deterministic, sometimes it takes just a couple
iterations, sometimes dozens.)
Ts is 2.0.0 that I compiled myself in debug mode on 64 bit linux.
I've run it through valgrind and haven't seen my plugin doing anything
suspicious - but I cannot reproduce the assert without the plugin
installed. The plugin does change the document size and content.
Unfortuantely, I don't own the copyright on the plugin code, so I cannot
post it in whole.
Anyone have any thoughts on what I might be doing wrong, or if there is
a caching problem related to adaptations that change document sizes?
FATAL: CacheDir.cc:693: failed assert `(unsigned int) dir_approx_size(dir) <= (unsigned int) (MAX_FRAG_SIZE + sizeofDoc)`
/home/mcmanus/ts/bin/traffic_server - STACK TRACE:
/home/mcmanus/ts/bin/traffic_server(ink_fatal_va+0xcb)[0x7579bb]
/home/mcmanus/ts/bin/traffic_server(ink_fatal+0xc7)[0x757aa5]
/home/mcmanus/ts/bin/traffic_server(_ink_assert+0xdb)[0x756bff]
/home/mcmanus/ts/bin/traffic_server(_Z13dir_overwriteP7INK_MD5P4PartP3DirS4_b+0x18d)[0x6e8838]
/home/mcmanus/ts/bin/traffic_server(_ZN7CacheVC22openWriteCloseHeadDoneEiP5Event+0x189)[0x70a2d1]
/home/mcmanus/ts/bin/traffic_server(_ZN12Continuation11handleEventEiPv+0x6c)[0x4f1c22]
/home/mcmanus/ts/bin/traffic_server(_ZN12PartCallback12aggWriteDoneEiP5Event+0x137)[0x7055dd]
/home/mcmanus/ts/bin/traffic_server(_ZN12Continuation11handleEventEiPv+0x6c)[0x4f1c22]
/home/mcmanus/ts/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x131)[0x74e66f]
/home/mcmanus/ts/bin/traffic_server(_ZN7EThread7executeEv+0x31e)[0x74eb12]
/home/mcmanus/ts/bin/traffic_server[0x74d1c6]
/lib/libpthread.so.0(+0x69ca)[0x7ffff7bc59ca]
/lib/libc.so.6(clone+0x6d)[0x7ffff59aa6fd]
Program received signal SIGABRT, Aborted.
[Switching to Thread 0x7ffff46a3710 (LWP 24261)]
0x00007ffff58f7a75 in *__GI_raise (sig=<value optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
64 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
in ../nptl/sysdeps/unix/sysv/linux/raise.c
(gdb) bt
#0 0x00007ffff58f7a75 in *__GI_raise (sig=<value optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
#1 0x00007ffff58fb5c0 in *__GI_abort () at abort.c:92
#2 0x00000000007578f0 in ink_die_die_die (retval=1) at ink_error.cc:43
#3 0x00000000007579c8 in ink_fatal_va (return_code=1,
message_format=0x7ffff46a29b0 "CacheDir.cc:693: failed assert `(unsigned int) dir_approx_size(dir) <= (unsigned int) (MAX_FRAG_SIZE + sizeofDoc)`", ap=0x7ffff46a28b0)
at ink_error.cc:67
#4 0x0000000000757aa5 in ink_fatal (return_code=1,
message_format=0x7ffff46a29b0 "CacheDir.cc:693: failed assert `(unsigned int) dir_approx_size(dir) <= (unsigned int) (MAX_FRAG_SIZE + sizeofDoc)`") at ink_error.cc:75
#5 0x0000000000756bff in _ink_assert (a=0x7ec9e0 "(unsigned int) dir_approx_size(dir) <= (unsigned int) (MAX_FRAG_SIZE + sizeofDoc)", f=0x7ec64c "CacheDir.cc", l=693)
at ink_assert.cc:47
#6 0x00000000006e8838 in dir_overwrite (key=0x1c21fe0, d=0x19d04d0, dir=0x1c22010, overwrite=0x1c370b8, must_overwrite=false) at CacheDir.cc:693
#7 0x000000000070a2d1 in CacheVC::openWriteCloseHeadDone (this=0x1c21f90, event=3900, e=0x0) at CacheWrite.cc:1170
#8 0x00000000004f1c22 in Continuation::handleEvent (this=0x1c21f90, event=3900, data=0x0) at ../iocore/eventsystem/I_Continuation.h:147
#9 0x00000000007055dd in PartCallback::aggWriteDone (this=0x19cdda0, event=1, e=0x1bc95e0) at CacheWrite.cc:327
#10 0x00000000004f1c22 in Continuation::handleEvent (this=0x19cdda0, event=1, data=0x1bc95e0) at ../iocore/eventsystem/I_Continuation.h:147
#11 0x000000000074e66f in EThread::process_event (this=0x7ffff4aa8010, e=0x1bc95e0, calling_code=1) at UnixEThread.cc:113
#12 0x000000000074eb12 in EThread::execute (this=0x7ffff4aa8010) at UnixEThread.cc:219
#13 0x000000000074d1c6 in spawn_thread_internal (a=0xc090c0) at Thread.cc:85
#14 0x00007ffff7bc59ca in start_thread (arg=<value optimized out>) at pthread_create.c:300
#15 0x00007ffff59aa6fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#16 0x0000000000000000 in ?? ()
(gdb)
Re: ts 2.0.0 failed assert `FATAL: CacheDir.cc:693
Posted by Patrick McManus <mc...@ducksong.com>.
On Wed, 2010-08-11 at 17:51 -0400, Patrick McManus wrote:
> On Wed, 2010-08-11 at 13:27 -0700, John Plevyak wrote:
> > I have created a bug:
> >
> > https://issues.apache.org/jira/secure/ManageAttachments.jspa?id=12471346
> >
> >
> > There is a patch attached. See if that fixes your problem.
> >
> > john
>
> John,
>
> A million thank yous for looking at the issue. Definite progress, but it
> now assert()s in a different place. Seemingly on the cache read path -
>
>
> FATAL: CacheRead.cc:981: failed assert `f.single_segment`
On closer inspection there is a valid path for !f.single_segment - we
just need to update the patch to take that now that we have violated the
assumption.
Does this updated patch make sense to you? It passes my tests with
flying colors..
diff --git a/iocore/cache/CacheRead.cc b/iocore/cache/CacheRead.cc
index 09195d8..3a0bad7 100644
--- a/iocore/cache/CacheRead.cc
+++ b/iocore/cache/CacheRead.cc
@@ -978,10 +978,12 @@ CacheVC::openReadStartHead(int event, Event * e)
doc_len = alternate.object_size_get();
if (key == doc->key) { // is this my data?
f.single_segment = doc->single_segment();
- ink_assert(f.single_segment); // otherwise need to read earliest
- ink_assert(doc->hlen);
- docpos = sizeofDoc + doc->hlen;
- next_CacheKey(&key, &doc->key);
+ if (f.single_segment)
+ {
+ ink_assert(doc->hlen);
+ docpos = sizeofDoc + doc->hlen;
+ next_CacheKey(&key, &doc->key);
+ }
} else {
f.single_segment = false;
}
diff --git a/iocore/cache/CacheWrite.cc b/iocore/cache/CacheWrite.cc
index a1f529a..8e59db1 100644
--- a/iocore/cache/CacheWrite.cc
+++ b/iocore/cache/CacheWrite.cc
@@ -191,10 +191,14 @@ CacheVC::handleWrite(int event, Event * e)
vec_len = 0;
set_agg_write_in_progress();
POP_HANDLER;
- agg_len = round_to_approx_size(write_len + vec_len + sizeofDoc);
+ int to_write = write_len + vec_len + sizeofDoc;
+ if (to_write > MAX_FRAG_SIZE) {
+ write_len = MAX_FRAG_SIZE - vec_len - sizeofDoc;
+ to_write = MAX_FRAG_SIZE;
+ }
+ agg_len = round_to_approx_size(to_write);
part->agg_todo_size += agg_len;
- ink_assert(agg_len <= AGG_SIZE);
- bool agg_error = (agg_len > AGG_SIZE ||
+ bool agg_error = (agg_len > AGG_SIZE || write_len < 0 ||
(!f.readers && (part->agg_todo_size > cache_config_agg_write_backlog + AGG_SIZE) && write_len));
#ifdef CACHE_AGG_FAIL_RATE
agg_error = agg_error || ((inku32) mutex->thread_holding->generator.random() <
@@ -211,6 +215,7 @@ CacheVC::handleWrite(int event, Event * e)
io.aio_result = AIO_SOFT_FAILURE;
return handleEvent(AIO_EVENT_DONE, 0);
}
+ ink_assert(agg_len <= AGG_SIZE);
if (f.evac_vector)
part->agg.push(this);
else
Re: ts 2.0.0 failed assert `FATAL: CacheDir.cc:693
Posted by Patrick McManus <mc...@ducksong.com>.
On Wed, 2010-08-11 at 13:27 -0700, John Plevyak wrote:
> I have created a bug:
>
> https://issues.apache.org/jira/secure/ManageAttachments.jspa?id=12471346
>
>
> There is a patch attached. See if that fixes your problem.
>
> john
John,
A million thank yous for looking at the issue. Definite progress, but it
now assert()s in a different place. Seemingly on the cache read path -
FATAL: CacheRead.cc:981: failed assert `f.single_segment`
/home/mcmanus/ts/bin/traffic_server - STACK TRACE:
/home/mcmanus/ts/bin/traffic_server(ink_fatal_va+0xcb)[0x7579ef]
/home/mcmanus/ts/bin/traffic_server(ink_fatal+0xc7)[0x757ad9]
/home/mcmanus/ts/bin/traffic_server(_ink_assert+0xdb)[0x756c33]
/home/mcmanus/ts/bin/traffic_server(_ZN7CacheVC17openReadStartHeadEiP5Event+0x956)[0x7031a0]
/home/mcmanus/ts/bin/traffic_server(_ZN12Continuation11handleEventEiPv+0x6c)[0x4f1c22]
/home/mcmanus/ts/bin/traffic_server(_ZN7CacheVC14handleReadDoneEiP5Event+0x92c)[0x6db656]
/home/mcmanus/ts/bin/traffic_server(_ZN12Continuation11handleEventEiPv+0x6c)[0x4f1c22]
/home/mcmanus/ts/bin/traffic_server(_ZN19AIOCallbackInternal11io_completeEiPv+0x3f)[0x6e2691]
/home/mcmanus/ts/bin/traffic_server(_ZN12Continuation11handleEventEiPv+0x6c)[0x4f1c22]
/home/mcmanus/ts/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x131)[0x74e6a3]
/home/mcmanus/ts/bin/traffic_server(_ZN7EThread7executeEv+0x99)[0x74e8c1]
/home/mcmanus/ts/bin/traffic_server[0x74d1fa]
/lib/libpthread.so.0(+0x69ca)[0x7ffff7bc59ca]
/lib/libc.so.6(clone+0x6d)[0x7ffff59aa6fd]
Program received signal SIGABRT, Aborted.
[Switching to Thread 0x7ffff46a3710 (LWP 14525)]
0x00007ffff58f7a75 in *__GI_raise (sig=<value optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
64 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
in ../nptl/sysdeps/unix/sysv/linux/raise.c
(gdb) bt
#0 0x00007ffff58f7a75 in *__GI_raise (sig=<value optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
#1 0x00007ffff58fb5c0 in *__GI_abort () at abort.c:92
#2 0x0000000000757924 in ink_die_die_die (retval=1) at ink_error.cc:43
#3 0x00000000007579fc in ink_fatal_va (return_code=1, message_format=0x7ffff46a2820 "CacheRead.cc:981: failed assert `f.single_segment`", ap=0x7ffff46a2720) at ink_error.cc:67
#4 0x0000000000757ad9 in ink_fatal (return_code=1, message_format=0x7ffff46a2820 "CacheRead.cc:981: failed assert `f.single_segment`") at ink_error.cc:75
#5 0x0000000000756c33 in _ink_assert (a=0x7f2a3e "f.single_segment", f=0x7f226b "CacheRead.cc", l=981) at ink_assert.cc:47
#6 0x00000000007031a0 in CacheVC::openReadStartHead (this=0xf6f4a0, event=3900, e=0x0) at CacheRead.cc:981
#7 0x00000000004f1c22 in Continuation::handleEvent (this=0xf6f4a0, event=3900, data=0x0) at ../iocore/eventsystem/I_Continuation.h:147
#8 0x00000000006db656 in CacheVC::handleReadDone (this=0xf6f4a0, event=3900, e=0xf6f618) at Cache.cc:1885
#9 0x00000000004f1c22 in Continuation::handleEvent (this=0xf6f4a0, event=3900, data=0xf6f618) at ../iocore/eventsystem/I_Continuation.h:147
#10 0x00000000006e2691 in AIOCallbackInternal::io_complete (this=0xf6f618, event=1, data=0xf0db30) at ../../iocore/aio/P_AIO.h:82
#11 0x00000000004f1c22 in Continuation::handleEvent (this=0xf6f618, event=1, data=0xf0db30) at ../iocore/eventsystem/I_Continuation.h:147
#12 0x000000000074e6a3 in EThread::process_event (this=0x7ffff4aa8010, e=0xf0db30, calling_code=1) at UnixEThread.cc:113
#13 0x000000000074e8c1 in EThread::execute (this=0x7ffff4aa8010) at UnixEThread.cc:168
#14 0x000000000074d1fa in spawn_thread_internal (a=0xc090c0) at Thread.cc:85
#15 0x00007ffff7bc59ca in start_thread (arg=<value optimized out>) at pthread_create.c:300
#16 0x00007ffff59aa6fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#17 0x0000000000000000 in ?? ()
(gdb)
Re: ts 2.0.0 failed assert `FATAL: CacheDir.cc:693
Posted by John Plevyak <jp...@apache.org>.
I have created a bug:
https://issues.apache.org/jira/secure/ManageAttachments.jspa?id=12471346
There is a patch attached. See if that fixes your problem.
john
On 8/11/2010 11:09 AM, Patrick McManus wrote:
> On Wed, 2010-08-11 at 13:53 -0400, Patrick McManus wrote:
>> John, thanks for the idea.
>
> Some more information
>
> * The test in question reduces the document from 281KB to 259KB - the
> headers are not altered.
>
> * The test just runs curl serially fetching the same URL (with the same
> transform) 750 times.. the assert() will fail in the first few dozen
> iterations.
>
> * I can now see a repeatable timeline:
>
> + transaction N goes through INK_EVENT_VCONN_WRITE_COMPLETE (having
> sent all its transformed data to the user agent)
> + transaction N+1 is started for the same URL as N
> + transaction N receives INK_EVENT_HTTP_TXN_CLOSE
> + assert() fails
>
> It appears from the logs that if N and N+1 do not overlap there is no
> assert failure.
>
Re: ts 2.0.0 failed assert `FATAL: CacheDir.cc:693
Posted by John Plevyak <jp...@apache.org>.
OK, this tells me what I need to know. The problem is that
the cache is not assuming that you will dump more than
250K or so into a buffer before you call into the cache.
The cache can handle long documents, but it assumes that
you will hand them over piecemeal in smaller chunks.
I am going to create a bug and see if I can get you a
real patch.
john
On 8/11/2010 11:09 AM, Patrick McManus wrote:
> On Wed, 2010-08-11 at 13:53 -0400, Patrick McManus wrote:
>> John, thanks for the idea.
>
> Some more information
>
> * The test in question reduces the document from 281KB to 259KB - the
> headers are not altered.
>
> * The test just runs curl serially fetching the same URL (with the same
> transform) 750 times.. the assert() will fail in the first few dozen
> iterations.
>
> * I can now see a repeatable timeline:
>
> + transaction N goes through INK_EVENT_VCONN_WRITE_COMPLETE (having
> sent all its transformed data to the user agent)
> + transaction N+1 is started for the same URL as N
> + transaction N receives INK_EVENT_HTTP_TXN_CLOSE
> + assert() fails
>
> It appears from the logs that if N and N+1 do not overlap there is no
> assert failure.
>
Re: ts 2.0.0 failed assert `FATAL: CacheDir.cc:693
Posted by Patrick McManus <mc...@ducksong.com>.
On Wed, 2010-08-11 at 13:53 -0400, Patrick McManus wrote:
> John, thanks for the idea.
Some more information
* The test in question reduces the document from 281KB to 259KB - the
headers are not altered.
* The test just runs curl serially fetching the same URL (with the same
transform) 750 times.. the assert() will fail in the first few dozen
iterations.
* I can now see a repeatable timeline:
+ transaction N goes through INK_EVENT_VCONN_WRITE_COMPLETE (having
sent all its transformed data to the user agent)
+ transaction N+1 is started for the same URL as N
+ transaction N receives INK_EVENT_HTTP_TXN_CLOSE
+ assert() fails
It appears from the logs that if N and N+1 do not overlap there is no
assert failure.
Re: ts 2.0.0 failed assert `FATAL: CacheDir.cc:693
Posted by Patrick McManus <mc...@ducksong.com>.
John, thanks for the idea.
Here is what happened -
First, I just changed the test in CacheWrite.cc as you said. I wasn't
clear on whether the assert() should now be satisfied or not without
moving it.. the test continued to fail in the same way.
Then I moved the assert from cachedir to cachewrite as you indicated -
the test runs a bit longer subjectively, but it fails on the (new)
assert.
I don't yet understand the flow of the cache code.
This is the patch::
diff --git a/iocore/cache/CacheDir.cc b/iocore/cache/CacheDir.cc
index 2cb271f..28e98f6 100644
--- a/iocore/cache/CacheDir.cc
+++ b/iocore/cache/CacheDir.cc
@@ -690,7 +690,7 @@ dir_overwrite(CacheKey * key, Part * d, Dir * dir, Dir * overwrite, bool must_ov
check_dir(d);
#endif
- ink_assert((unsigned int) dir_approx_size(dir) <= (unsigned int) (MAX_FRAG_SIZE + sizeofDoc)); // XXX - size should be unsigned
+
Lagain:
// find entry to overwrite
e = b;
diff --git a/iocore/cache/CacheWrite.cc b/iocore/cache/CacheWrite.cc
index a1f529a..1e0966a 100644
--- a/iocore/cache/CacheWrite.cc
+++ b/iocore/cache/CacheWrite.cc
@@ -791,6 +791,8 @@ agg_copy(char *p, CacheVC * vc)
ink_assert(part_offset(part, &vc->dir) < (part->skip + part->len));
dir_set_phase(&vc->dir, part->header->phase);
+ ink_assert((unsigned int) dir_approx_size(&vc->dir) <= (unsigned int) (MAX_FRAG_SIZE + sizeofDoc)); // XXX - size should be unsigned
+
// fill in document header
doc->magic = DOC_MAGIC;
doc->len = seglen;
@@ -1362,7 +1364,8 @@ Lagain:
total_len += avail;
}
length = towrite;
- if (length > TARGET_FRAG_SIZE && length < SHRINK_TARGET_FRAG_SIZE)
+ // if (length > TARGET_FRAG_SIZE && length < SHRINK_TARGET_FRAG_SIZE)
+ if (length > TARGET_FRAG_SIZE)
write_len = TARGET_FRAG_SIZE;
else
write_len = length;
Re: ts 2.0.0 failed assert `FATAL: CacheDir.cc:693
Posted by John Plevyak <jp...@apache.org>.
This is on 2.0. This is a very simple consistency check
of the cache directory. The assert is finding
that a fragment size estimate in the directory is greater
than the max possible. In 2.0 this is 256k. It doesn't
look like there is any code to prevent HTTP from passing
in a buffer larger than this, so perhaps that is what is
happening.
To test this find the lines:
if (length > TARGET_FRAG_SIZE && length < SHRINK_TARGET_FRAG_SIZE)
write_len = TARGET_FRAG_SIZE;
else
write_len = length;
And remove the second conditional:
if (length > TARGET_FRAG_SIZE)
write_len = TARGET_FRAG_SIZE;
else
write_len = length;
This presumes that you have a rational size header. If you header
is on the order of 200k then the problem might still occur.
To test both conditions take that assert from 693 and insert it
at CacheWrite.cc:792 after the dir_set_approx_size and pass it vc->dir.
This will verify that the size is correct when it is set in the directory.
Tell me how this works out and I'll try to formulate a patch.
john
On 8/11/2010 9:36 AM, Patrick McManus wrote:
> Hi all -
>
> I'm working on a trafficserver plugin - and if I spin my ~40 case test
> suite through TS repeatedly I can pretty easily reproduce the assert
> below. (It is not deterministic, sometimes it takes just a couple
> iterations, sometimes dozens.)
>
> Ts is 2.0.0 that I compiled myself in debug mode on 64 bit linux.
>
> I've run it through valgrind and haven't seen my plugin doing anything
> suspicious - but I cannot reproduce the assert without the plugin
> installed. The plugin does change the document size and content.
>
> Unfortuantely, I don't own the copyright on the plugin code, so I cannot
> post it in whole.
>
> Anyone have any thoughts on what I might be doing wrong, or if there is
> a caching problem related to adaptations that change document sizes?
>
> FATAL: CacheDir.cc:693: failed assert `(unsigned int) dir_approx_size(dir) <= (unsigned int) (MAX_FRAG_SIZE + sizeofDoc)`
> /home/mcmanus/ts/bin/traffic_server - STACK TRACE:
> /home/mcmanus/ts/bin/traffic_server(ink_fatal_va+0xcb)[0x7579bb]
> /home/mcmanus/ts/bin/traffic_server(ink_fatal+0xc7)[0x757aa5]
> /home/mcmanus/ts/bin/traffic_server(_ink_assert+0xdb)[0x756bff]
> /home/mcmanus/ts/bin/traffic_server(_Z13dir_overwriteP7INK_MD5P4PartP3DirS4_b+0x18d)[0x6e8838]
> /home/mcmanus/ts/bin/traffic_server(_ZN7CacheVC22openWriteCloseHeadDoneEiP5Event+0x189)[0x70a2d1]
> /home/mcmanus/ts/bin/traffic_server(_ZN12Continuation11handleEventEiPv+0x6c)[0x4f1c22]
> /home/mcmanus/ts/bin/traffic_server(_ZN12PartCallback12aggWriteDoneEiP5Event+0x137)[0x7055dd]
> /home/mcmanus/ts/bin/traffic_server(_ZN12Continuation11handleEventEiPv+0x6c)[0x4f1c22]
> /home/mcmanus/ts/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x131)[0x74e66f]
> /home/mcmanus/ts/bin/traffic_server(_ZN7EThread7executeEv+0x31e)[0x74eb12]
> /home/mcmanus/ts/bin/traffic_server[0x74d1c6]
> /lib/libpthread.so.0(+0x69ca)[0x7ffff7bc59ca]
> /lib/libc.so.6(clone+0x6d)[0x7ffff59aa6fd]
>
> Program received signal SIGABRT, Aborted.
> [Switching to Thread 0x7ffff46a3710 (LWP 24261)]
> 0x00007ffff58f7a75 in *__GI_raise (sig=<value optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
> 64 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
> in ../nptl/sysdeps/unix/sysv/linux/raise.c
> (gdb) bt
> #0 0x00007ffff58f7a75 in *__GI_raise (sig=<value optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
> #1 0x00007ffff58fb5c0 in *__GI_abort () at abort.c:92
> #2 0x00000000007578f0 in ink_die_die_die (retval=1) at ink_error.cc:43
> #3 0x00000000007579c8 in ink_fatal_va (return_code=1,
> message_format=0x7ffff46a29b0 "CacheDir.cc:693: failed assert `(unsigned int) dir_approx_size(dir) <= (unsigned int) (MAX_FRAG_SIZE + sizeofDoc)`", ap=0x7ffff46a28b0)
> at ink_error.cc:67
> #4 0x0000000000757aa5 in ink_fatal (return_code=1,
> message_format=0x7ffff46a29b0 "CacheDir.cc:693: failed assert `(unsigned int) dir_approx_size(dir) <= (unsigned int) (MAX_FRAG_SIZE + sizeofDoc)`") at ink_error.cc:75
> #5 0x0000000000756bff in _ink_assert (a=0x7ec9e0 "(unsigned int) dir_approx_size(dir) <= (unsigned int) (MAX_FRAG_SIZE + sizeofDoc)", f=0x7ec64c "CacheDir.cc", l=693)
> at ink_assert.cc:47
> #6 0x00000000006e8838 in dir_overwrite (key=0x1c21fe0, d=0x19d04d0, dir=0x1c22010, overwrite=0x1c370b8, must_overwrite=false) at CacheDir.cc:693
> #7 0x000000000070a2d1 in CacheVC::openWriteCloseHeadDone (this=0x1c21f90, event=3900, e=0x0) at CacheWrite.cc:1170
> #8 0x00000000004f1c22 in Continuation::handleEvent (this=0x1c21f90, event=3900, data=0x0) at ../iocore/eventsystem/I_Continuation.h:147
> #9 0x00000000007055dd in PartCallback::aggWriteDone (this=0x19cdda0, event=1, e=0x1bc95e0) at CacheWrite.cc:327
> #10 0x00000000004f1c22 in Continuation::handleEvent (this=0x19cdda0, event=1, data=0x1bc95e0) at ../iocore/eventsystem/I_Continuation.h:147
> #11 0x000000000074e66f in EThread::process_event (this=0x7ffff4aa8010, e=0x1bc95e0, calling_code=1) at UnixEThread.cc:113
> #12 0x000000000074eb12 in EThread::execute (this=0x7ffff4aa8010) at UnixEThread.cc:219
> #13 0x000000000074d1c6 in spawn_thread_internal (a=0xc090c0) at Thread.cc:85
> #14 0x00007ffff7bc59ca in start_thread (arg=<value optimized out>) at pthread_create.c:300
> #15 0x00007ffff59aa6fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
> #16 0x0000000000000000 in ?? ()
> (gdb)
>