You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@jackrabbit.apache.org by mr...@apache.org on 2019/05/20 11:23:20 UTC
svn commit: r1859550 [18/26] - in /jackrabbit/site/live/archive: ./ wiki/
wiki/JCR/ wiki/JCR/attachments/ wiki/JCR/attachments/115513387/
wiki/JCR/attachments/115513390/ wiki/JCR/attachments/115513408/
wiki/JCR/attachments/115513413/ wiki/JCR/attachmen...
Added: jackrabbit/site/live/archive/wiki/JCR/Oakathon-March-2018_115513477.html
URL: http://svn.apache.org/viewvc/jackrabbit/site/live/archive/wiki/JCR/Oakathon-March-2018_115513477.html?rev=1859550&view=auto
==============================================================================
--- jackrabbit/site/live/archive/wiki/JCR/Oakathon-March-2018_115513477.html (added)
+++ jackrabbit/site/live/archive/wiki/JCR/Oakathon-March-2018_115513477.html Mon May 20 11:23:18 2019
@@ -0,0 +1,441 @@
+<!DOCTYPE html>
+<html>
+ <head>
+ <title>Apache Jackrabbit : Oakathon March 2018</title>
+ <link rel="stylesheet" href="styles/site.css" type="text/css" />
+ <META http-equiv="Content-Type" content="text/html; charset=UTF-8">
+ </head>
+
+ <body class="theme-default aui-theme-default">
+ <div id="page">
+ <div id="main" class="aui-page-panel">
+ <div id="main-header">
+ <div id="breadcrumb-section">
+ <ol id="breadcrumbs">
+ <li class="first">
+ <span><a href="index.html">Apache Jackrabbit</a></span>
+ </li>
+ <li>
+ <span><a href="Home_70731.html">Home</a></span>
+ </li>
+ </ol>
+ </div>
+ <h1 id="title-heading" class="pagetitle">
+ <span id="title-text">
+ Apache Jackrabbit : Oakathon March 2018
+ </span>
+ </h1>
+ </div>
+
+ <div id="content" class="view">
+ <div class="page-metadata">
+
+
+
+
+
+
+
+ Created by <span class='author'> ASF Infrabot</span> on May 20, 2019
+ </div>
+ <div id="main-content" class="wiki-content group">
+ <style type='text/css'>/*<![CDATA[*/
+div.rbtoc1558350682906 {padding: 0px;}
+div.rbtoc1558350682906 ul {list-style: disc;margin-left: 0px;}
+div.rbtoc1558350682906 li {margin-left: 0px;padding-left: 0px;}
+
+/*]]>*/</style><div class='toc-macro rbtoc1558350682906'>
+<ul class='toc-indentation'>
+<li><a href='#OakathonMarch2018-WhereandWhen'>Where and When</a></li>
+<li><a href='#OakathonMarch2018-Attendees'>Attendees</a></li>
+<li><a href='#OakathonMarch2018-Topics/Discussions/Goals'>Topics/Discussions/Goals</a></li>
+<li><a href='#OakathonMarch2018-AgendaProposal'>Agenda Proposal</a></li>
+<li><a href='#OakathonMarch2018-PrepWork'>Prep Work</a></li>
+<li><a href='#OakathonMarch2018-NotesfromtheOakathon'>Notes from the Oakathon</a>
+<ul class='toc-indentation'>
+<li><a href='#OakathonMarch2018-DocumentMKRoadmap'>DocumentMK Roadmap</a></li>
+<li><a href='#OakathonMarch2018-BlobIDMapping'>CompositeDataStore Blob ID Mapping</a></li>
+<li><a href='#OakathonMarch2018-Modularisation'>Modularisation</a>
+<ul class='toc-indentation'>
+<li><a href='#OakathonMarch2018-WhatisanOakrelease?'>What is an Oak release ?</a></li>
+<li><a href='#OakathonMarch2018-HowdowedeliverOakupdates?'>How do we deliver Oak updates?</a></li>
+<li><a href='#OakathonMarch2018-Wheredowekeepmodules?'>Where do we keep modules?</a></li>
+<li><a href='#OakathonMarch2018-Nextsteps'>Next steps</a></li>
+</ul>
+</li>
+</ul>
+</li>
+</ul>
+</div>
+
+<h1 id="OakathonMarch2018-WhereandWhen">Where and When</h1>
+<ul>
+ <li>March 19th - 23rd 2018</li>
+ <li>Location: <a href="https://goo.gl/maps/7sMbCv4HsXT2" class="external-link" rel="nofollow">Adobe Basel</a> (Meeting Room Rhein, 5th floor), guests please register at reception on 2nd floor.
+<br class="atl-forced-newline"/></li>
+</ul>
+
+
+<h1 id="OakathonMarch2018-Attendees">Attendees</h1>
+<div class="table-wrap"><table class="confluenceTable"><tbody>
+<tr>
+<td class="confluenceTd"><p> Who </p></td>
+<td class="confluenceTd"><p> When </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p>Julian Reschke </p></td>
+<td class="confluenceTd"><p> 19. and 22./23. (remotely) </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p>Marcel Reutegger </p></td>
+<td class="confluenceTd"><p> 19. - 22. </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p>Matt Ryan </p></td>
+<td class="confluenceTd"><p> 19. - 23. </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p>Robert Munteanu </p></td>
+<td class="confluenceTd"><p> 20. - 22. </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p>Thomas Mueller </p></td>
+<td class="confluenceTd"><p> 20. - 23. </p></td>
+</tr>
+</tbody></table></div>
+
+
+<h1 id="OakathonMarch2018-Topics/Discussions/Goals">Topics/Discussions/Goals</h1>
+<div class="table-wrap"><table class="confluenceTable"><tbody>
+<tr>
+<td class="confluenceTd"><p> Title </p></td>
+<td class="confluenceTd"><p> Summary </p></td>
+<td class="confluenceTd"><p> Effort </p></td>
+<td class="confluenceTd"><p> Participants </p></td>
+<td class="confluenceTd"><p> Proposed by </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p> Avoid reads from MongoDB primary </p></td>
+<td class="confluenceTd"><p> Work on a prototype using MongoDB 3.6 client sessions with causal consistency. See <a href="https://issues.apache.org/jira/browse/OAK-6087" class="external-link" rel="nofollow">OAK-6087</a>. </p></td>
+<td class="confluenceTd"><p> 2-3 days </p></td>
+<td class="confluenceTd"><p>Â </p></td>
+<td class="confluenceTd"><p> Marcel Reutegger </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p> <a class="createlink" href="/confluence/pages/createpage.action?spaceKey=JCR&title=CompositeDataStore&linkCreation=true&fromPageId=115513477">CompositeDataStore</a> - How to map blob IDs to delegates </p></td>
+<td class="confluenceTd"><p> Revisit discussion on how to map blob IDs to <a class="createlink" href="/confluence/pages/createpage.action?spaceKey=JCR&title=CompositeDataStore&linkCreation=true&fromPageId=115513477">CompositeDataStore</a> delegates. Proposals include: maintaining <a href="https://en.wikipedia.org/wiki/Perfect_hash_function#Minimal_perfect_hash_function" class="external-link" rel="nofollow">MPHF</a> tables or Bloom filters to map IDs to delegates, rebuilding them every time data store GC is run or on some other schedule; encoding a delegate identifier into the blob ID; ??? </p></td>
+<td class="confluenceTd"><p> 2 hours to discuss; we could also try to prototype the solution (2-3 days). </p></td>
+<td class="confluenceTd"><p> Matt Ryan, Amit Jain, Thomas Mueller, Chetan Mehrotra </p></td>
+<td class="confluenceTd"><p> Matt Ryan </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p> <a class="createlink" href="/confluence/pages/createpage.action?spaceKey=JCR&title=CompositeDataStore&linkCreation=true&fromPageId=115513477">CompositeDataStore</a> - review pull request </p></td>
+<td class="confluenceTd"><p> The first version of the <a class="createlink" href="/confluence/pages/createpage.action?spaceKey=JCR&title=CompositeDataStore&linkCreation=true&fromPageId=115513477">CompositeDataStore</a> is in pull request and ready to review - can we go through it and resolve the issues? </p></td>
+<td class="confluenceTd"><p> 2 hours </p></td>
+<td class="confluenceTd"><p> Matt Ryan, Amit Jain, Chetan Mehrotra, Thomas Mueller </p></td>
+<td class="confluenceTd"><p> Matt Ryan </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p> DocumentMK Roadmap </p></td>
+<td class="confluenceTd"><p> Development focus for 2018 </p></td>
+<td class="confluenceTd"><p> 0.5h </p></td>
+<td class="confluenceTd"><p>Â </p></td>
+<td class="confluenceTd"><p> Marcel Reutegger </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p> <a href="DataStore_115513387.html">DataStore</a> Roadmap </p></td>
+<td class="confluenceTd"><p> Changes needed for better container support </p></td>
+<td class="confluenceTd"><p> 0.5h </p></td>
+<td class="confluenceTd"><p>Â </p></td>
+<td class="confluenceTd"><p> Matt Ryan </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p> Modularization </p></td>
+<td class="confluenceTd"><p> Where do we stand for modularization and where to go next </p></td>
+<td class="confluenceTd"><p> 1h </p></td>
+<td class="confluenceTd"><p>Â </p></td>
+<td class="confluenceTd"><p> Robert Munteanu</p></td>
+</tr>
+</tbody></table></div>
+
+
+<h1 id="OakathonMarch2018-AgendaProposal">Agenda Proposal</h1>
+
+<div class="table-wrap"><table class="confluenceTable"><tbody>
+<tr>
+<td class="confluenceTd"><p>Mon</p></td>
+<td class="confluenceTd"><p>Â </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p> 9:00-12:30</p></td>
+<td class="confluenceTd"><p>9:00 Setup<br/>
+9:30 5-10 mins overview per topic<br/>
+10:00 DocumentMK Roadmap</p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p>13:30-17:00</p></td>
+<td class="confluenceTd"><p>Â </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p>Â </p></td>
+<td class="confluenceTd"><p>Â </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p>Tue</p></td>
+<td class="confluenceTd"><p>Â </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p> 9:00-12:30</p></td>
+<td class="confluenceTd"><p> 9:30 TarMK Roadmap<br/>
+10:30 <a class="createlink" href="/confluence/pages/createpage.action?spaceKey=JCR&title=CompositeDataStore&linkCreation=true&fromPageId=115513477">CompositeDataStore</a> BlobID mapping</p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p>13:30-17:00</p></td>
+<td class="confluenceTd"><p> 13:30 - 15:30 Disposable Publishers</p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p>Â </p></td>
+<td class="confluenceTd"><p>Â </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p>Wed</p></td>
+<td class="confluenceTd"><p>Â </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p> 9:00-12:30</p></td>
+<td class="confluenceTd"><p> 9:30 Indexing Roadmap</p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p>13:30-17:00</p></td>
+<td class="confluenceTd"><p>Â </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p>Â </p></td>
+<td class="confluenceTd"><p>Â </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p>Thu</p></td>
+<td class="confluenceTd"><p>Â </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p> 9:00-12:30</p></td>
+<td class="confluenceTd"><p> 9:30 Modularization </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p>13:30-17:00</p></td>
+<td class="confluenceTd"><p>Â </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p>Â </p></td>
+<td class="confluenceTd"><p>Â </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p>Fri</p></td>
+<td class="confluenceTd"><p>Â </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p> 9:00-12:30</p></td>
+<td class="confluenceTd"><p>Â </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p>13:30-17:00</p></td>
+<td class="confluenceTd"><p>Â </p></td>
+</tr>
+</tbody></table></div>
+
+
+<h1 id="OakathonMarch2018-PrepWork">Prep Work</h1>
+
+
+<h1 id="OakathonMarch2018-NotesfromtheOakathon">Notes from the Oakathon</h1>
+
+<ul>
+ <li>Add documentation how to run integration tests on different backend. E.g. Postgres.
+<br class="atl-forced-newline"/></li>
+</ul>
+
+
+<h2 id="OakathonMarch2018-DocumentMKRoadmap">DocumentMK Roadmap</h2>
+
+<ul>
+ <li>Cloud deployments</li>
+ <li>Areas of improvement</li>
+ <li>Roundtrips
+ <ul>
+ <li>Need to pay for each call</li>
+ <li>Review and reduce communication with service</li>
+ </ul>
+ </li>
+ <li>Monitoring
+ <ul>
+ <li>More metrics</li>
+ <li>Document existing metrics (meaning, thresholds)</li>
+ </ul>
+ </li>
+ <li>DevOps
+ <ul>
+ <li>Avoid manual tasks</li>
+ <li>Upgrade/rollback</li>
+ <li>Compatibility</li>
+ <li>Tooling (machine readable output, versions, compatibility)</li>
+ </ul>
+ </li>
+ <li>Deployment
+ <ul>
+ <li>Container ready (local disk?, persistent cache)</li>
+ <li>Provisioned DocumentStore ops (something in OSGi?)</li>
+ <li>Independent maintenance tasks (serverless)</li>
+ </ul>
+ </li>
+ <li>Resilience
+ <ul>
+ <li>Lease timeout behaviour</li>
+ <li>Incorrect/erratic clock</li>
+ <li>Unit tests / integration tests / gap! / longevity tests</li>
+ <li>Analyze reported issues</li>
+ </ul>
+ </li>
+ <li>Alternative back-ends
+ <ul>
+ <li>Cosmos DB
+<br class="atl-forced-newline"/></li>
+ </ul>
+ </li>
+</ul>
+
+
+<h2 id="OakathonMarch2018-BlobIDMapping"><a class="createlink" href="/confluence/pages/createpage.action?spaceKey=JCR&title=CompositeDataStore&linkCreation=true&fromPageId=115513477">CompositeDataStore</a> Blob ID Mapping</h2>
+<p>Since the <a class="createlink" href="/confluence/pages/createpage.action?spaceKey=JCR&title=CompositeDataStore&linkCreation=true&fromPageId=115513477">CompositeDataStore</a> contains multiple delegate data stores inside it, an obvious question is: How does it know which delegate to choose for any particular record?</p>
+
+<p>There are two sub-questions:</p>
+<ul>
+ <li>Is it required to implement a blob ID mapping in order to accept the current <a class="createlink" href="/confluence/pages/createpage.action?spaceKey=JCR&title=CompositeDataStore&linkCreation=true&fromPageId=115513477">CompositeDataStore</a> pull requests (<a href="https://github.com/apache/jackrabbit-oak/pull/71" class="external-link" rel="nofollow">1</a>, <a href="https://github.com/apache/jackrabbit-oak/pull/74" class="external-link" rel="nofollow">2</a>), or can we accept them into Oak without this capability in order to get testing started using <a class="createlink" href="/confluence/pages/createpage.action?spaceKey=JCR&title=CompositeDataStore&linkCreation=true&fromPageId=115513477">CompositeDataStore</a>, and add the mapping later?</li>
+ <li>What technique should we use to do map blob IDs to delegates? Some of the ideas proposed include:
+ <ul>
+ <li>Encoding the delegate into the blob ID.
+ <ul>
+ <li>Challenges:
+ <ul>
+ <li>Other <a class="createlink" href="/confluence/pages/createpage.action?spaceKey=JCR&title=CompositeDataStore&linkCreation=true&fromPageId=115513477">CompositeDataStore</a> scenarios like storage tiering (automatically moving blobs to cheaper storage) would then require re-encoding the blob ID each time the data is moved from one delegate to another.</li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ <li>Maintaining a mapping data structure.
+ <ul>
+ <li>Proposed ideas include <a href="https://en.wikipedia.org/wiki/Perfect_hash_function#Minimal_perfect_hash_function" class="external-link" rel="nofollow">MPHF</a> or Bloom filters (both proposed by @Thomas)
+ <ul>
+ <li>Challenges:
+ <ul>
+ <li>Initial creation of table
+ <ul>
+ <li>Build anew each startup or save it somewhere and reload?</li>
+ </ul>
+ </li>
+ <li>Resizing the table
+ <ul>
+ <li>Adjust as needed or wait until a certain event (e.g. data store GC times)?
+ <ul>
+ <li>What if user never runs data store GC?</li>
+ </ul>
+ </li>
+ <li>How to maintain state in the meantime?
+<br class="atl-forced-newline"/></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+</ul>
+
+
+<p>Decisions that came out of the session:</p>
+<ul>
+ <li>Some minimal amount of testing needs to be done to quantify the performance impact of accepting the current pull requests without doing some sort of mapping. This needs to be known before we can make a decision on accepting the pull request.</li>
+ <li>To address the blob ID mapping question, the decision was to first try to implement an encoding of a data store identifier into the blob ID. Since we don't have any current requirements to support moving of blobs from one data store delegate to another (and since in the last Oakathon we made a similar decision that this was beyond the responsibilities of Oak anyway), we won't use that as a reason to not encode the data store ID into the blob ID. If subsequently we need to move beyond those capabilities we can additionally add an in-memory mapping of some sort that would be used first, before looking at the blob ID, as an override for the data store location.</li>
+ <li>The implementation and use case needs to be more fully documented within oak-doc.
+<br class="atl-forced-newline"/></li>
+</ul>
+
+
+<h2 id="OakathonMarch2018-Modularisation">Modularisation</h2>
+
+<p>We discussed what the end goal is for the Oak modularisation. For 1.8 we split out many modules, which are now more cohesive and we have more fine-grained depedencies. Aiming for independendent module releases presents other challenges we have not discussed yet.</p>
+
+<h3 id="OakathonMarch2018-WhatisanOakrelease?">What is an Oak release ?</h3>
+
+<p>Up till now we have released all modules in lock-step, which means that there is a very clear definition of what an Oak release is. With modules released independently it's not clear what modules should be used by consumers. This basically affects:</p>
+
+<ul>
+ <li>consumers using Oak in their product (Sling/AEM)</li>
+ <li>oak-run, as the tools that embeds many Oak modules
+<br class="atl-forced-newline"/></li>
+</ul>
+
+
+<p>Consumers already ask about which dependency of oak-run goes with which Oak version, this will get more complicated with modular releases. </p>
+
+<p>A number of ideas were raised for simplifying oak-run management:</p>
+
+<ul>
+ <li>specify dependencies (using CLI/files) - downside is that it's tedious and requires network access</li>
+ <li>using jar files from an existing Oak/Sling/AEM installation - this can be done, but it requires reading from an exploded Felix container bundle state, not very stable. Also no guarantee that all oak-run dependencies are present ( e.g. CLI parsing code usually not in Sling/AEM )</li>
+ <li>generate a new version of Oak-run ( fat jar or script ) whenever a specific oak bundle is started - cleaner version of the above, with the same note that some dependencies might not be present. We can add the missing dependencies to the deployment, given that they don't have a large disk footprint.
+<br class="atl-forced-newline"/></li>
+</ul>
+
+
+<p>ASF release note: if needed, we can release multiple modules in one vote, e.g. oak-segment-tar and oak-run.</p>
+
+<h3 id="OakathonMarch2018-HowdowedeliverOakupdates?">How do we deliver Oak updates?</h3>
+
+<p>Now consumers update all Oak bundles together. Upgrading just parts of the bundles can lead to confusion, e.g. "What Oak version am I on?".</p>
+
+<h3 id="OakathonMarch2018-Wheredowekeepmodules?">Where do we keep modules?</h3>
+
+<p>Once extracting a module, we need to find a proper place for it. The options we discussed were</p>
+
+<ul>
+ <li><a href="https://svn.apache.org/repos/asf/jackrabbit/$" class="external-link" rel="nofollow">https://svn.apache.org/repos/asf/jackrabbit/$\</a>{module}</li>
+ <li><a href="https://svn.apache.org/repos/asf/jackrabbit/modules/$" class="external-link" rel="nofollow">https://svn.apache.org/repos/asf/jackrabbit/modules/$\</a>{module}</li>
+ <li>a separate git repository
+<br class="atl-forced-newline"/></li>
+</ul>
+
+
+<p>We did not reach a conclusion so modules will live in the same SVN location, just removed from the main reactor POM ( see also Next steps below ).</p>
+
+<h3 id="OakathonMarch2018-Nextsteps">Next steps</h3>
+
+<p>To minimise risk and get a better understanding of the implications of independent release we will start off with isolated modules and give them an independent release cycles. Good candidates would:</p>
+
+<ul>
+ <li>be leaves in the dependency tree - no other modules depend on them</li>
+ <li>have no SNAPSHOT dependencies</li>
+ <li>have few dependencies</li>
+ <li>have only versioned OSGi imports
+<br class="atl-forced-newline"/></li>
+</ul>
+
+
+<p>One clear candidate is the oak-blob-cloud-azure module.</p>
+
+<p>The simplest way to cut it from the release is to remove it from the reactor pom and keep it in the same location in SVN. The project will be built and imported into IDEs independently.</p>
+ </div>
+
+
+
+ </div> </div>
+ <div id="footer" role="contentinfo">
+ <section class="footer-body">
+ <p>Document generated by Confluence on May 20, 2019 11:11</p>
+ <div id="footer-logo"><a href="http://www.atlassian.com/">Atlassian</a></div>
+ </section>
+ </div>
+ </div> </body>
+</html>
Propchange: jackrabbit/site/live/archive/wiki/JCR/Oakathon-March-2018_115513477.html
------------------------------------------------------------------------------
svn:eol-style = native
Propchange: jackrabbit/site/live/archive/wiki/JCR/Oakathon-March-2018_115513477.html
------------------------------------------------------------------------------
svn:executable = *
Added: jackrabbit/site/live/archive/wiki/JCR/Oakathon-May-2019_115513478.html
URL: http://svn.apache.org/viewvc/jackrabbit/site/live/archive/wiki/JCR/Oakathon-May-2019_115513478.html?rev=1859550&view=auto
==============================================================================
--- jackrabbit/site/live/archive/wiki/JCR/Oakathon-May-2019_115513478.html (added)
+++ jackrabbit/site/live/archive/wiki/JCR/Oakathon-May-2019_115513478.html Mon May 20 11:23:18 2019
@@ -0,0 +1,271 @@
+<!DOCTYPE html>
+<html>
+ <head>
+ <title>Apache Jackrabbit : Oakathon May 2019</title>
+ <link rel="stylesheet" href="styles/site.css" type="text/css" />
+ <META http-equiv="Content-Type" content="text/html; charset=UTF-8">
+ </head>
+
+ <body class="theme-default aui-theme-default">
+ <div id="page">
+ <div id="main" class="aui-page-panel">
+ <div id="main-header">
+ <div id="breadcrumb-section">
+ <ol id="breadcrumbs">
+ <li class="first">
+ <span><a href="index.html">Apache Jackrabbit</a></span>
+ </li>
+ <li>
+ <span><a href="Home_70731.html">Home</a></span>
+ </li>
+ </ol>
+ </div>
+ <h1 id="title-heading" class="pagetitle">
+ <span id="title-text">
+ Apache Jackrabbit : Oakathon May 2019
+ </span>
+ </h1>
+ </div>
+
+ <div id="content" class="view">
+ <div class="page-metadata">
+
+
+
+
+
+
+
+ Created by <span class='author'> ASF Infrabot</span> on May 20, 2019
+ </div>
+ <div id="main-content" class="wiki-content group">
+ <style type='text/css'>/*<![CDATA[*/
+div.rbtoc1558350682978 {padding: 0px;}
+div.rbtoc1558350682978 ul {list-style: disc;margin-left: 0px;}
+div.rbtoc1558350682978 li {margin-left: 0px;padding-left: 0px;}
+
+/*]]>*/</style><div class='toc-macro rbtoc1558350682978'>
+<ul class='toc-indentation'>
+<li><a href='#OakathonMay2019-WhereandWhen'>Where and When</a></li>
+<li><a href='#OakathonMay2019-Attendees'>Attendees</a></li>
+<li><a href='#OakathonMay2019-Topics/Discussions/Goals'>Topics/Discussions/Goals</a></li>
+<li><a href='#OakathonMay2019-AgendaProposal'>Agenda Proposal</a>
+<ul class='toc-indentation'>
+<li><a href='#OakathonMay2019-Monday'>Monday</a></li>
+<li><a href='#OakathonMay2019-Tuesday'>Tuesday</a></li>
+<li><a href='#OakathonMay2019-Wednesday'>Wednesday</a></li>
+<li><a href='#OakathonMay2019-Thursday'>Thursday</a></li>
+<li><a href='#OakathonMay2019-Friday'>Friday</a></li>
+</ul>
+</li>
+<li><a href='#OakathonMay2019-PrepWork'>Prep Work</a>
+<ul class='toc-indentation'>
+<li><a href='#OakathonMay2019-Buildtimeouts'>Build timeouts</a></li>
+<li><a href='#OakathonMay2019-Jackrabbit'>Jackrabbit</a></li>
+</ul>
+</li>
+<li><a href='#OakathonMay2019-NotesfromtheOakathon'>Notes from the Oakathon</a>
+<ul class='toc-indentation'>
+<li><a href='#OakathonMay2019-MigrateWiki'>Migrate Wiki</a></li>
+<li><a href='#OakathonMay2019-CI/CD'>CI/CD</a></li>
+<li><a href='#OakathonMay2019-AsyncCommits'>Async Commits</a></li>
+<li><a href='#OakathonMay2019-GuavaUpdate'>Guava Update</a></li>
+</ul>
+</li>
+</ul>
+</div>
+
+<h1 id="OakathonMay2019-WhereandWhen">Where and When</h1>
+<ul>
+ <li>Date: May 13th - 17th</li>
+ <li>Location: Adobe Basel, Barfüsserplatz 6 (guests please register on the 2nd floor)</li>
+ <li>Meeting Rooms: Rhein (5th floor, except Friday on 3rd floor)
+<br class="atl-forced-newline"/></li>
+</ul>
+
+
+<h1 id="OakathonMay2019-Attendees">Attendees</h1>
+<div class="table-wrap"><table class="confluenceTable"><tbody>
+<tr>
+<td class="confluenceTd"><p> Who </p></td>
+<td class="confluenceTd"><p> When </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p> Matt Ryan </p></td>
+<td class="confluenceTd"><p> 14.5.19 - 17.5.19 </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p> Marcel Reutegger </p></td>
+<td class="confluenceTd"><p> 13.5.19 - 16.5.19 </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p> Julian Reschke </p></td>
+<td class="confluenceTd"><p> 13.5.19 - 17.5.19 (remote)</p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p> Stefan Egli </p></td>
+<td class="confluenceTd"><p> 13.5.19 - 16.5.19 (onsite except 15.5.)</p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p> Alex D </p></td>
+<td class="confluenceTd"><p> 13 - 16 (remote 17)</p></td>
+</tr>
+</tbody></table></div>
+
+
+<h1 id="OakathonMay2019-Topics/Discussions/Goals">Topics/Discussions/Goals</h1>
+<div class="table-wrap"><table class="confluenceTable"><tbody>
+<tr>
+<td class="confluenceTd"><p> Title </p></td>
+<td class="confluenceTd"><p> Summary </p></td>
+<td class="confluenceTd"><p> Effort </p></td>
+<td class="confluenceTd"><p> Participants </p></td>
+<td class="confluenceTd"><p> Proposed by </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p> Build timeouts </p></td>
+<td class="confluenceTd"><p> Jenkins builds hit the 2h timeout every now and then. Discuss options to reduce the build time and implement. </p></td>
+<td class="confluenceTd"><p> 2d </p></td>
+<td class="confluenceTd"><p> Julian, Marcel </p></td>
+<td class="confluenceTd"><p> Marcel </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p> Migrate Wiki </p></td>
+<td class="confluenceTd"><p> The Moin Wiki goes out of service by the end of May. Migrate to Confluence. </p></td>
+<td class="confluenceTd"><p>Â </p></td>
+<td class="confluenceTd"><p> Marcel, Davide </p></td>
+<td class="confluenceTd"><p> Matt </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p> Direct Binary Access API </p></td>
+<td class="confluenceTd"><p> Some early uses of the Direct Binary Access feature hint at possible API changes. Since the current use is pretty minimal and changing the API in the future will be much more painful, should we consider some of these changes now? (Matt to prepare use cases for discussion.) </p></td>
+<td class="confluenceTd"><p> 1h discussion / 1-2 days to implement? </p></td>
+<td class="confluenceTd"><p> Matt </p></td>
+<td class="confluenceTd"><p> Matt </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p> Managing Expectations for Oak Changes </p></td>
+<td class="confluenceTd"><p> We've recently changed our release cadence. I'd like to discuss setting and managing appropriate expectations for outside parties wrt how long it will take to see bug fixes and new features in an Oak release. </p></td>
+<td class="confluenceTd"><p> 30m - 1h </p></td>
+<td class="confluenceTd"><p> Matt </p></td>
+<td class="confluenceTd"><p> Matt </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p> CI/CD </p></td>
+<td class="confluenceTd"><p> How can allow developers to run their own CI/CD pipeline with additional integration tests that are not part of Oak? </p></td>
+<td class="confluenceTd"><p> 2h </p></td>
+<td class="confluenceTd"><p>Â </p></td>
+<td class="confluenceTd"><p> Marcel </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p> Jackrabbit </p></td>
+<td class="confluenceTd"><p> Branch or backport API changes in Jackrabbit for Oak 1.14.0 </p></td>
+<td class="confluenceTd"><p> 15m </p></td>
+<td class="confluenceTd"><p>Â </p></td>
+<td class="confluenceTd"><p> Julian </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p> Async Commits </p></td>
+<td class="confluenceTd"><p> Discuss feasibility of revisiting async conflict resolution (</p>
+<a href="https://oak.markmail.org/thread/p2ljif2vag3alktf" class="external-link" rel="nofollow">https://oak.markmail.org/thread/p2ljif2vag3alktf</a>
+<p>) which could be used to implement async commits </p></td>
+<td class="confluenceTd"><p> 2d </p></td>
+<td class="confluenceTd"><p>Â </p></td>
+<td class="confluenceTd"><p> Stefan </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p> Oakathon Cadence and Purpose </p></td>
+<td class="confluenceTd"><p> Discussion about Oakathon frequency, and to ensure we maintain the purpose and spirit of Oakathons moving forward. </p></td>
+<td class="confluenceTd"><p> 1h </p></td>
+<td class="confluenceTd"><p> All interested </p></td>
+<td class="confluenceTd"><p> Matt </p></td>
+</tr>
+</tbody></table></div>
+
+
+<h1 id="OakathonMay2019-AgendaProposal">Agenda Proposal</h1>
+
+<h2 id="OakathonMay2019-Monday">Monday</h2>
+
+<ul>
+ <li>11:00 Kick off. Short intro of proposed topics and form groups.
+<br class="atl-forced-newline"/></li>
+</ul>
+
+
+<h2 id="OakathonMay2019-Tuesday">Tuesday</h2>
+<ul>
+ <li>9:45 Intro to remaining topics proposed by Matt</li>
+ <li>13:15 Managing Expectations for Oak Changes</li>
+</ul>
+
+
+<h2 id="OakathonMay2019-Wednesday">Wednesday</h2>
+
+<ul>
+ <li>9:30 Oakathon Cadence and Purpose</li>
+ <li>13:30 CI/CD
+<br class="atl-forced-newline"/></li>
+</ul>
+
+
+<h2 id="OakathonMay2019-Thursday">Thursday</h2>
+
+<h2 id="OakathonMay2019-Friday">Friday</h2>
+
+<h1 id="OakathonMay2019-PrepWork">Prep Work</h1>
+
+<h2 id="OakathonMay2019-Buildtimeouts">Build timeouts</h2>
+
+<p>The intro for this topic went a bit beyond the proposed problem. There are various other Oak build jobs on Apache Jenkins. Julian is taking care of a number of matrix jobs that are failing rather frequently but do not send notifications to the list. We agreed to update the documentation with links to the existing jobs and look into making them more stable. Some fail with docker errors. The plan is to ask Apache Infra what Jenkins slaves actually support Docker and are recommended to use.</p>
+
+<p>Reducing build time could then be achieved by splitting the whole build into a matrix that runs modules individually and in parallel.</p>
+
+<h2 id="OakathonMay2019-Jackrabbit">Jackrabbit</h2>
+
+<p>When Oak needs a new Jackrabbit API, there are basically two options on the table. 1) backport the change to the current stable branch or 2) create a new stable release from trunk. Currently that would be a 2.20.</p>
+
+<p>Discussion started about pros and cons. Going with a new stable release from trunk would mean a new release policy that first has to be discussed on the Jackrabbit dev list. There also the open question whether a new stable release from Jackrabbit trunk means a new maintenance branch has to be created. For Oak we decided this is something we don't do anymore automatically, unless needed because of backward incompatible changes. There was a concern that this may result in frequent Jackrabbit stable releases whenever the API must be enhanced for Oak. Though, last year, this was only needed once for the new direct binary access feature. Alternatively we may consider moving the Jackrabbit API to Oak.</p>
+
+<h1 id="OakathonMay2019-NotesfromtheOakathon">Notes from the Oakathon</h1>
+
+<h2 id="OakathonMay2019-MigrateWiki">Migrate Wiki</h2>
+
+<p>The consensus of the group was to not migrate the Moin wiki to Confluence and instead just create a browsable archive of the existing Moin wiki. The archive would then be available as part of the Jackrabbit website. The archived wiki should have a disclaimer that those pages are not meant as documentation and link to the official documentation. The current Confluence wiki contains an outdated version of the Jackrabbit site and will be deleted.</p>
+
+<p>The Confluence wiki pages that represented an old version of the Jackrabbit website are now deleted. Marcel did a test migration of the Moin to the Confluence wiki with the help of <a href="http://selfserve.apache.org/" class="external-link" rel="nofollow">http://selfserve.apache.org/</a></p>
+
+<p>Further progress/work is tracked in <a href="https://issues.apache.org/jira/browse/JCR-4435" class="external-link" rel="nofollow">https://issues.apache.org/jira/browse/JCR-4435</a></p>
+
+<h2 id="OakathonMay2019-CI/CD">CI/CD</h2>
+
+<p>One option for running additional non-Apache integration tests would be to fork the Jackrabbit Oak <a class="createlink" href="/confluence/pages/createpage.action?spaceKey=JCR&title=GitHub&linkCreation=true&fromPageId=115513478">GitHub</a> mirror, create feature branches and work on those branches until a change is ready. Then a PR back to the fork could trigger additional integration tests that act as an additional quality gate until the change is committed back to the Apache SVN. The last step would still be manual. The development workflow can potentially be simplified if the Oak code base was on <a class="createlink" href="/confluence/pages/createpage.action?spaceKey=JCR&title=GitHub&linkCreation=true&fromPageId=115513478">GitHub</a>. Then it might be possible to further automate the process by automatically triggering a PR against the Oak origin on <a class="createlink" href="/confluence/pages/createpage.action?spaceKey=JCR&title=GitHub&linkCr
eation=true&fromPageId=115513478">GitHub</a> when the fork applied a validated PR with the additional external integration tests. The final PR obviously would still have to pass all Oak unit and integration tests.</p>
+
+<p>Moving the code base to <a class="createlink" href="/confluence/pages/createpage.action?spaceKey=JCR&title=GitHub&linkCreation=true&fromPageId=115513478">GitHub</a> seems to make it easier for a developer even if there are some manual steps needed to get a change into Oak. It would make the use of git-svn obsolete. </p>
+
+<h2 id="OakathonMay2019-AsyncCommits">Async Commits</h2>
+<ul>
+ <li>Created <a href="https://issues.apache.org/jira/browse/OAK-8326" class="external-link" rel="nofollow">https://issues.apache.org/jira/browse/OAK-8326</a> to track this</li>
+ <li>A sketch of a prototype is done in <a href="https://github.com/stefan-egli/jackrabbit-oak/tree/OAK-8326" class="external-link" rel="nofollow">https://github.com/stefan-egli/jackrabbit-oak/tree/OAK-8326</a></li>
+ <li>A description of the sketch is available <a href="https://issues.apache.org/jira/browse/OAK-8326?focusedCommentId=16841399&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16841399" class="external-link" rel="nofollow">here</a>
+<br class="atl-forced-newline"/>
+<br class="atl-forced-newline"/></li>
+</ul>
+
+
+<h2 id="OakathonMay2019-GuavaUpdate">Guava Update</h2>
+
+<p>Spent some time on "Guava references in public apis (<a href="https://issues.apache.org/jira/browse/OAK-7217" class="external-link" rel="nofollow">OAK-7217</a>)" as it seems to be blocking the Guava update decisions (<a href="https://issues.apache.org/jira/browse/OAK-7182" class="external-link" rel="nofollow">OAK-7182</a>). Tried a bash script but wasn't enough as it provided general "use of Guava apis" but not focused on oak apis, so changed gears and decided to go for an oak-run approach. this seemed a better option as it allows for solving a more generic issue of "foreign apis exported by oak apis" and it turns out we also expose commons-io objects (but also a lot of others). Latest version is <a href="https://issues.apache.org/jira/browse/OAK-7217?focusedCommentId=16841259&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16841259" class="external-link" rel="nofollow">attached to the issue</a>.</p>
+ </div>
+
+
+
+ </div> </div>
+ <div id="footer" role="contentinfo">
+ <section class="footer-body">
+ <p>Document generated by Confluence on May 20, 2019 11:11</p>
+ <div id="footer-logo"><a href="http://www.atlassian.com/">Atlassian</a></div>
+ </section>
+ </div>
+ </div> </body>
+</html>
Propchange: jackrabbit/site/live/archive/wiki/JCR/Oakathon-May-2019_115513478.html
------------------------------------------------------------------------------
svn:eol-style = native
Propchange: jackrabbit/site/live/archive/wiki/JCR/Oakathon-May-2019_115513478.html
------------------------------------------------------------------------------
svn:executable = *
Added: jackrabbit/site/live/archive/wiki/JCR/Oakathon-November-2017_115513479.html
URL: http://svn.apache.org/viewvc/jackrabbit/site/live/archive/wiki/JCR/Oakathon-November-2017_115513479.html?rev=1859550&view=auto
==============================================================================
--- jackrabbit/site/live/archive/wiki/JCR/Oakathon-November-2017_115513479.html (added)
+++ jackrabbit/site/live/archive/wiki/JCR/Oakathon-November-2017_115513479.html Mon May 20 11:23:18 2019
@@ -0,0 +1,464 @@
+<!DOCTYPE html>
+<html>
+ <head>
+ <title>Apache Jackrabbit : Oakathon November 2017</title>
+ <link rel="stylesheet" href="styles/site.css" type="text/css" />
+ <META http-equiv="Content-Type" content="text/html; charset=UTF-8">
+ </head>
+
+ <body class="theme-default aui-theme-default">
+ <div id="page">
+ <div id="main" class="aui-page-panel">
+ <div id="main-header">
+ <div id="breadcrumb-section">
+ <ol id="breadcrumbs">
+ <li class="first">
+ <span><a href="index.html">Apache Jackrabbit</a></span>
+ </li>
+ <li>
+ <span><a href="Home_70731.html">Home</a></span>
+ </li>
+ </ol>
+ </div>
+ <h1 id="title-heading" class="pagetitle">
+ <span id="title-text">
+ Apache Jackrabbit : Oakathon November 2017
+ </span>
+ </h1>
+ </div>
+
+ <div id="content" class="view">
+ <div class="page-metadata">
+
+
+
+
+
+
+
+ Created by <span class='author'> ASF Infrabot</span> on May 20, 2019
+ </div>
+ <div id="main-content" class="wiki-content group">
+ <style type='text/css'>/*<![CDATA[*/
+div.rbtoc1558350683084 {padding: 0px;}
+div.rbtoc1558350683084 ul {list-style: disc;margin-left: 0px;}
+div.rbtoc1558350683084 li {margin-left: 0px;padding-left: 0px;}
+
+/*]]>*/</style><div class='toc-macro rbtoc1558350683084'>
+<ul class='toc-indentation'>
+<li><a href='#OakathonNovember2017-WhereandWhen'>Where and When</a></li>
+<li><a href='#OakathonNovember2017-Attendees'>Attendees</a></li>
+<li><a href='#OakathonNovember2017-Topics/Discussions/Goals'>Topics/Discussions/Goals</a></li>
+<li><a href='#OakathonNovember2017-AgendaProposal'>Agenda Proposal</a></li>
+<li><a href='#OakathonNovember2017-PrepWork'>Prep Work</a></li>
+<li><a href='#OakathonNovember2017-NotesfromtheOakathon'>Notes from the Oakathon</a>
+<ul class='toc-indentation'>
+<li><a href='#OakathonNovember2017-ProvidingJCRNodeinformationto'>Providing JCR Node information to DataStore</a></li>
+<li><a href='#OakathonNovember2017-Handlingrequestsforblobsthataren'timmediatelyavailable'>Handling requests for blobs that aren't immediately available</a></li>
+</ul>
+</li>
+</ul>
+</div>
+
+<h1 id="OakathonNovember2017-WhereandWhen">Where and When</h1>
+<ul>
+ <li>November 13th - 17th 2017</li>
+ <li>Location: <a href="https://goo.gl/maps/7sMbCv4HsXT2" class="external-link" rel="nofollow">Adobe Basel</a>, guests please register at reception on 2nd floor. Oakathon will be on 5th floor in meeting room 'Rhein'.
+<br class="atl-forced-newline"/></li>
+</ul>
+
+
+<h1 id="OakathonNovember2017-Attendees">Attendees</h1>
+<div class="table-wrap"><table class="confluenceTable"><tbody>
+<tr>
+<td class="confluenceTd"><p> Who </p></td>
+<td class="confluenceTd"><p> When </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p>Marcel Reutegger </p></td>
+<td class="confluenceTd"><p> 13. - 16. </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p>Matt Ryan </p></td>
+<td class="confluenceTd"><p> 13. - 16. </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p>Angela Schreiber </p></td>
+<td class="confluenceTd"><p> 13. - 16. </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p>Michael Dürig </p></td>
+<td class="confluenceTd"><p> 13. - 16. </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p>Valentin Olteanu </p></td>
+<td class="confluenceTd"><p> 13. - 16. </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p>Andrei Dulceanu </p></td>
+<td class="confluenceTd"><p> 14. - 16. </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p>Francesco Mari </p></td>
+<td class="confluenceTd"><p> 13. - 16. </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p>Thomas Müller </p></td>
+<td class="confluenceTd"><p> 13. - 16. </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p>Daniel Hasler </p></td>
+<td class="confluenceTd"><p> 13. - 16. </p></td>
+</tr>
+</tbody></table></div>
+
+
+<h1 id="OakathonNovember2017-Topics/Discussions/Goals">Topics/Discussions/Goals</h1>
+<div class="table-wrap"><table class="confluenceTable"><tbody>
+<tr>
+<td class="confluenceTd"><p> Title </p></td>
+<td class="confluenceTd"><p> Summary </p></td>
+<td class="confluenceTd"><p> Effort </p></td>
+<td class="confluenceTd"><p> Participants </p></td>
+<td class="confluenceTd"><p> Proposed by </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p>Handling requests for blobs that aren't immediately available</p></td>
+<td class="confluenceTd"><p>Some cloud storage options, like AWS Glacier, offer the capability to store infrequently used data for a very low cost. The tradeoff is that requests for data may take a long time - even hours or longer - before the data is accessible. The purpose of the discussion will be to determine how to handle both the initial request and subsequent requests and how to help clients interact on such requests effectively. Having someone familiar with Sling involved in the discussion would be useful as Sling's request model may also be affected.</p></td>
+<td class="confluenceTd"><p>2h</p></td>
+<td class="confluenceTd"><p>everyone interested</p></td>
+<td class="confluenceTd"><p>Matt Ryan</p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p>Providing JCR node information at blob creation time</p></td>
+<td class="confluenceTd"><p>The JCR specification defines that values are created via 'ValueFactory.createBinary(stream)' (section 10.4.3.2). This is carried forward into the <a href="DataStore_115513387.html">DataStore</a> interface, wherein the addRecord() method takes only an input stream and has no JCR node information related to the stream. With the introduction of the <a class="createlink" href="/confluence/pages/createpage.action?spaceKey=JCR&title=CompositeDataStore&linkCreation=true&fromPageId=115513479">CompositeDataStore</a> into Oak, there are many interesting use cases that can potentially be supported if certain JCR node information (e.g. JCR path) is known when a binary is being created or accessed. This discussion is around ways to support these types of use cases.</p></td>
+<td class="confluenceTd"><p>2h (discussion only), or 1-2d (if we do a prototype</p></td>
+<td class="confluenceTd"><p>everyone interested</p></td>
+<td class="confluenceTd"><p>Matt Ryan</p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p> Threat Model </p></td>
+<td class="confluenceTd"><p> Create a threat model for Oak </p></td>
+<td class="confluenceTd"><p> 2 x 2hours </p></td>
+<td class="confluenceTd"><p> everyone interested; 1 expert per topic needed on demand </p></td>
+<td class="confluenceTd"><p> angela </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p> m12n </p></td>
+<td class="confluenceTd"><p> Continue/complete modularization effort </p></td>
+<td class="confluenceTd"><p> as long as it takes </p></td>
+<td class="confluenceTd"><p> everyone interested </p></td>
+<td class="confluenceTd"><p> angela </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p> TarMK tooling </p></td>
+<td class="confluenceTd"><p> Review and advance tooling API and initial implementations from the August Oakathon. </p></td>
+<td class="confluenceTd"><p> 2-5d </p></td>
+<td class="confluenceTd"><p> everyone interested </p></td>
+<td class="confluenceTd"><p> Michael </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p> Versioning and adoption </p></td>
+<td class="confluenceTd"><p> probably related to m12n, discuss how we think about Oak adoption when new feature are implemented, but upstream modules cannot take unstable versions </p></td>
+<td class="confluenceTd"><p> 2h </p></td>
+<td class="confluenceTd"><p> everyone interested </p></td>
+<td class="confluenceTd"><p> Alex </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p> TarMK roadmap </p></td>
+<td class="confluenceTd"><p> Sketch out a roadmap for the TarMK for the upcoming months. </p></td>
+<td class="confluenceTd"><p> 1d </p></td>
+<td class="confluenceTd"><p> Andrei, Francesco, Valentin, Michael and everyone interested </p></td>
+<td class="confluenceTd"><p> Michael </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p> TarMK pain points </p></td>
+<td class="confluenceTd"><p> Based on the current feedback, write down the list of (major) issues encountered by users when operating a TarMK deployment. Identify the main focus areas and prioritize to help defining the roadmap. </p></td>
+<td class="confluenceTd"><p> 2-3h </p></td>
+<td class="confluenceTd"><p> Andrei, Francesco, Valentin, Michael and everyone interested </p></td>
+<td class="confluenceTd"><p> Valentin </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p> TarMK on HDFS </p></td>
+<td class="confluenceTd"><p> Could it be possible to store segments on HDFS instead of a local disk? From a quick analysis this could be easier the intuitively perceived. HDFS scales exceptionally for parallel reads and writes of blocks. </p></td>
+<td class="confluenceTd"><p> 2-3d </p></td>
+<td class="confluenceTd"><p> Tomek, Francesco, Michael, Andrei and everyone interested </p></td>
+<td class="confluenceTd"><p> <a class="createlink" href="/confluence/pages/createpage.action?spaceKey=JCR&title=PhilippSuter&linkCreation=true&fromPageId=115513479">PhilippSuter</a> </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p> No page caching for TarMK </p></td>
+<td class="confluenceTd"><p> Page caching produces notorious side effects, especially for storing very large repositories. Could it be possible to use JVM managed memory to achieve similar cache hit ratios? </p></td>
+<td class="confluenceTd"><p> 2-3d </p></td>
+<td class="confluenceTd"><p> Andrei, Valentin, Francesco, Michael and everyone interested </p></td>
+<td class="confluenceTd"><p> <a class="createlink" href="/confluence/pages/createpage.action?spaceKey=JCR&title=PhilippSuter&linkCreation=true&fromPageId=115513479">PhilippSuter</a> </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p> In or out? </p></td>
+<td class="confluenceTd"><p> Go through <a href="https://issues.apache.org/jira/issues/?jql=project%20%3D%20OAK%20AND%20resolution%20%3D%20Unresolved%20AND%20fixVersion%20in%20(1.7.11%2C%201.7.12%2C%201.7.13%2C%201.7.14%2C%201.7.15%2C%201.8)" class="external-link" rel="nofollow">open issues</a> and decide what goes into 1.8 and what needs to be deferred. Committers familiar with a module should do a first pass before the Oakathon and use the time with the team to discuss issues that are controversial, on a tight schedule or require attention for some other reason. </p></td>
+<td class="confluenceTd"><p> 2-4h </p></td>
+<td class="confluenceTd"><p> everyone </p></td>
+<td class="confluenceTd"><p> Marcel </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p> Wrap up <a class="createlink" href="/confluence/pages/createpage.action?spaceKey=JCR&title=CompositeDataStore&linkCreation=true&fromPageId=115513479">CompositeDataStore</a> </p></td>
+<td class="confluenceTd"><p> I think <a class="createlink" href="/confluence/pages/createpage.action?spaceKey=JCR&title=CompositeDataStore&linkCreation=true&fromPageId=115513479">CompositeDataStore</a> is almost across the finish line, let's tie a bow on it </p></td>
+<td class="confluenceTd"><p> 2-4h </p></td>
+<td class="confluenceTd"><p> MattR + 1-2 committers familiar with data store </p></td>
+<td class="confluenceTd"><p> Matt Ryan </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p> Benchmarking </p></td>
+<td class="confluenceTd"><p> Writing benchmarks and interpreting results is challenging. I would like to present and discuss problems, solutions, and possible improvements. See also: <a href="https://blog.acolyer.org/2017/11/06/statistically-rigorous-java-performance-evaluation/" class="external-link" rel="nofollow">Statistically rigorous Java performance evaluation</a>, <a href="https://blog.acolyer.org/2017/11/07/virtual-machine-warmup-blows-hot-and-cold/" class="external-link" rel="nofollow">Virtual machine warmup blows hot and cold</a>, <a href="http://bravenewgeek.com/everything-you-know-about-latency-is-wrong/" class="external-link" rel="nofollow">Everything You Know About Latency Is Wrong</a> </p></td>
+<td class="confluenceTd"><p> 1h </p></td>
+<td class="confluenceTd"><p> everyone interested </p></td>
+<td class="confluenceTd"><p> Thomas </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p> Serverless Computing, first for Index+Search </p></td>
+<td class="confluenceTd"><p> Serverless computing is gaining traction. See also: <a href="https://blog.acolyer.org/2017/10/19/serverless-computing-economic-and-architectural-impact/" class="external-link" rel="nofollow">Serverless computing: economic and architectural impact</a>. We should discuss in which modules it can be used. One example is Search and Queries </p></td>
+<td class="confluenceTd"><p> 1h </p></td>
+<td class="confluenceTd"><p> everyone interested </p></td>
+<td class="confluenceTd"><p> Thomas </p></td>
+</tr>
+</tbody></table></div>
+
+
+
+<h1 id="OakathonNovember2017-AgendaProposal">Agenda Proposal</h1>
+
+<div class="table-wrap"><table class="confluenceTable"><tbody>
+<tr>
+<td class="confluenceTd"><p>Mon</p></td>
+<td class="confluenceTd"><p>General</p></td>
+<td class="confluenceTd"><p>DataStore</p></td>
+<td class="confluenceTd"><p>TarMK</p></td>
+<td class="confluenceTd"><p>Misc</p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p>9:00-12:30</p></td>
+<td class="confluenceTd"><p>9:00 Setup<br/>
+9:30 5 min overview per topic</p></td>
+</tr>
+</tbody></table></div>
+
+
+<p>10:30 Providing JCR node information at blob creation time|</p>
+
+<p>10:30 TarMK pain points| |</p>
+<div class="table-wrap"><table class="confluenceTable"><tbody>
+<tr>
+<td class="confluenceTd"><p>13:30-17:00</p></td>
+<td class="confluenceTd"><p>Â </p></td>
+<td class="confluenceTd"><p>Â </p></td>
+<td class="confluenceTd"><p>14:00 TarMK pain points </p></td>
+<td class="confluenceTd"><p>13:30 Modularization</p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p>Â </p></td>
+<td class="confluenceTd"><p>Â </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p>Tue</p></td>
+<td class="confluenceTd"><p>Â </p></td>
+<td class="confluenceTd"><p>Â </p></td>
+<td class="confluenceTd"><p>Â </p></td>
+<td class="confluenceTd"><p>Â </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p>9:00-12:30</p></td>
+<td class="confluenceTd"><p>9:00 Benchmarking (1h)</p></td>
+</tr>
+<tr>
+</tr>
+</tbody></table></div>
+
+<p>10:00 Handling requests for blobs that aren't immediately available |TarMK tooling| |</p>
+<div class="table-wrap"><table class="confluenceTable"><tbody>
+<tr>
+<td class="confluenceTd"><p>13:30-17:00</p></td>
+<td class="confluenceTd"><p>Â </p></td>
+<td class="confluenceTd"><p>Wrap up <a class="createlink" href="/confluence/pages/createpage.action?spaceKey=JCR&title=CompositeDataStore&linkCreation=true&fromPageId=115513479">CompositeDataStore</a></p></td>
+</tr>
+</tbody></table></div>
+
+<p>15:00 Threat modeling (2h)|</p>
+<div class="table-wrap"><table class="confluenceTable"><tbody>
+<tr>
+<td class="confluenceTd"><p>Â </p></td>
+<td class="confluenceTd"><p>Â </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p>Wed</p></td>
+<td class="confluenceTd"><p>Â </p></td>
+<td class="confluenceTd"><p>Â </p></td>
+<td class="confluenceTd"><p>Â </p></td>
+<td class="confluenceTd"><p>Â </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p>9:00-12:30</p></td>
+<td class="confluenceTd"><p>9:00 Versioning and adoption (1h)</p></td>
+<td class="confluenceTd"><p>Â </p></td>
+<td class="confluenceTd"><p> 10:00 TarMK future</p></td>
+</tr>
+</tbody></table></div>
+
+<p>10:00 Threat modeling (2h) |</p>
+<div class="table-wrap"><table class="confluenceTable"><tbody>
+<tr>
+<td class="confluenceTd"><p>13:30-17:00</p></td>
+<td class="confluenceTd"><p>Â </p></td>
+<td class="confluenceTd"><p>Â </p></td>
+<td class="confluenceTd"><p>13:30 TarMK future <br/>
+ 16:00 Wrap up: TarMK future</p></td>
+<td class="confluenceTd"><p>Â </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p>18:00-... </p></td>
+<td class="confluenceTd"><p>Dinner. Separate invite pending </p></td>
+<td class="confluenceTd"><p>Â </p></td>
+<td class="confluenceTd"><p>Â </p></td>
+<td class="confluenceTd"><p>Â </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p>Â </p></td>
+<td class="confluenceTd"><p>Â </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p>Thu</p></td>
+<td class="confluenceTd"><p>Â </p></td>
+<td class="confluenceTd"><p>Â </p></td>
+<td class="confluenceTd"><p>Â </p></td>
+<td class="confluenceTd"><p>Â </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p>9:00-12:30</p></td>
+<td class="confluenceTd"><p>9:00 In or out? (2h)</p></td>
+<td class="confluenceTd"><p>Â </p></td>
+<td class="confluenceTd"><p>09:00 TarMK on HDFS (Tomek) <br/>
+ 10:00 No page caching for TarMK</p></td>
+<td class="confluenceTd"><p>Â </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p>13:30-17:00</p></td>
+<td class="confluenceTd"><p>13:30 Serverless Computing, first for Indexing+Search(1h) <br/>
+ 14:30 Oak Future - Where is Oak headed? (1.5h) </p></td>
+<td class="confluenceTd"><p>Â </p></td>
+<td class="confluenceTd"><p>Â </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p>Â </p></td>
+<td class="confluenceTd"><p>Â </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p>Fri</p></td>
+<td class="confluenceTd"><p>Â </p></td>
+<td class="confluenceTd"><p>Â </p></td>
+<td class="confluenceTd"><p>Â </p></td>
+<td class="confluenceTd"><p>Â </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p>9:00-12:30</p></td>
+<td class="confluenceTd"><p>Â </p></td>
+<td class="confluenceTd"><p>Â </p></td>
+<td class="confluenceTd"><p>Â </p></td>
+<td class="confluenceTd"><p>Â </p></td>
+</tr>
+<tr>
+<td class="confluenceTd"><p>13:30-17:00</p></td>
+<td class="confluenceTd"><p>Â </p></td>
+<td class="confluenceTd"><p>Â </p></td>
+<td class="confluenceTd"><p>Â </p></td>
+<td class="confluenceTd"><p>Â </p></td>
+</tr>
+</tbody></table></div>
+
+
+<h1 id="OakathonNovember2017-PrepWork">Prep Work</h1>
+
+<h1 id="OakathonNovember2017-NotesfromtheOakathon">Notes from the Oakathon</h1>
+<h2 id="OakathonNovember2017-ProvidingJCRNodeinformationto">Providing JCR Node information to <a href="DataStore_115513387.html">DataStore</a></h2>
+<p>There are two main cases to consider: Creating a new blob and accessing an existing blob.</p>
+<ul>
+ <li>Blob creation is done in the <a href="DataStore_115513387.html">DataStore</a> interface via addRecord(InputStream). Options discussed (in order of preference) were:
+ <ul>
+ <li>Creating a new <a class="createlink" href="/confluence/pages/createpage.action?spaceKey=JCR&title=InputStream&linkCreation=true&fromPageId=115513479">InputStream</a> implementation that includes node information in the input stream. When the <a href="DataStore_115513387.html">DataStore</a> reads the stream if the stream is of the new implementation type it will pull the node information out of the stream and then send the rest along, or something like that.</li>
+ <li>Extend the Jackrabbit <a href="DataStore_115513387.html">DataStore</a> API to also support addRecord() with additional node information. This would not replace the addRecord(InputStream) method, but would be additive (and, we admit, not strictly compliant with the JCR spec).</li>
+ <li>Use an existing deprecated method that might suit this purpose. We dislike this, obviously, because we would be knowingly using a deprecated method.</li>
+ </ul>
+ </li>
+ <li>Accessing an existing blob is <em>usually</em> but not always via a <a class="createlink" href="/confluence/pages/createpage.action?spaceKey=JCR&title=DataIdentifier&linkCreation=true&fromPageId=115513479">DataIdentifier</a>.
+ <ul>
+ <li>We considered encoding additional information into the <a class="createlink" href="/confluence/pages/createpage.action?spaceKey=JCR&title=DataIdentifier&linkCreation=true&fromPageId=115513479">DataIdentifier</a>, but we are leaning away from that for a few reasons:
+ <ul>
+ <li>Encoding node information into the identifier means that blob ids would have to change if any of the node information were ever to change, like adding a property or moving the blob to a different path. It also presents complications for supporting binary deduplication (the same blob may be stored at two different paths).</li>
+ <li>Encoding a data store identifier into the blob id has similar issues if the blob were to be moved from one data store to another.</li>
+ <li>Taking this step also creates a data migration issue for existing users.</li>
+ </ul>
+ </li>
+ <li>Instead we discussed that the <a class="createlink" href="/confluence/pages/createpage.action?spaceKey=JCR&title=CompositeDataStore&linkCreation=true&fromPageId=115513479">CompositeDataStore</a> can assume the responsibility for mapping <a class="createlink" href="/confluence/pages/createpage.action?spaceKey=JCR&title=DataIdentifiers&linkCreation=true&fromPageId=115513479">DataIdentifiers</a> to delegate data stores. This was basically considered a requirement for <a class="createlink" href="/confluence/pages/createpage.action?spaceKey=JCR&title=CompositeDataStore&linkCreation=true&fromPageId=115513479">CompositeDataStore</a> anyway (via Bloom filters). The <a class="createlink" href="/confluence/pages/createpage.action?spaceKey=JCR&title=CompositeDataStore&linkCreation=true&fromPageId=115513479">CompositeDataStore</a> would need to load existing identifiers at startup time to do this. We might be able to get the <a class="cre
atelink" href="/confluence/pages/createpage.action?spaceKey=JCR&title=DataIdentifiers&linkCreation=true&fromPageId=115513479">DataIdentifiers</a> via the blob tracker.
+<br class="atl-forced-newline"/></li>
+ </ul>
+ </li>
+</ul>
+
+
+<p>During the meeting we also brought up a number of issues that need to be verified with <a class="createlink" href="/confluence/pages/createpage.action?spaceKey=JCR&title=CompositeDataStore&linkCreation=true&fromPageId=115513479">CompositeDataStore</a>, related to this topic:</p>
+<ul>
+ <li>We need to check initiation of <a class="createlink" href="/confluence/pages/createpage.action?spaceKey=JCR&title=CompositeDataStore&linkCreation=true&fromPageId=115513479">CompositeDataStore</a> in the system and make sure that caching gets initialized correctly.</li>
+ <li>We need to check DSGC in the production system/test system use case. Since the test system accesses the production data store read-only, does it also participate in the mark phase? If binaries are marked for delete in production, do they end up getting deleted from the production data store?</li>
+ <li>In the production system/test system use case, can the test system reuse or share the index segments from the production system, or is the test system required to rebuild the indexes for the test system use? This may take so long that it limits the usefulness of the test system, so this needs to be understood. How would this work if the production instance is doing active deletion of Lucene indexes?
+ <ul>
+ <li>Can we clone an instance and also clone the index segments if active deletion is being used? Since the clone only happens from the node store's head state, would the clone care about other information not at the head state?</li>
+ <li>Is it okay to have separate index segments between both (and rebuild them), or copy them and update them for the local system, or would it be better to try to share the index segments?
+<br class="atl-forced-newline"/></li>
+ </ul>
+ </li>
+</ul>
+
+
+<p>Finally, we discussed what we may consider to be the first use case of this capability in Oak. Initially Matt proposed that allowing the <a class="createlink" href="/confluence/pages/createpage.action?spaceKey=JCR&title=CompositeDataStore&linkCreation=true&fromPageId=115513479">CompositeDataStore</a> to select a delegate based on path information may be the first use case. Another suggestion (Amit? Vikas?) was that a smaller use case might exist just within Oak to use a <a class="createlink" href="/confluence/pages/createpage.action?spaceKey=JCR&title=CompositeDataStore&linkCreation=true&fromPageId=115513479">CompositeDataStore</a> and store only index segments in one delegate and everything else in the other. In that case this would happen entirely within Oak and the user would not be aware that a <a class="createlink" href="/confluence/pages/createpage.action?spaceKey=JCR&title=CompositeDataStore&linkCreation=true&fromPageId=115513479">Com
positeDataStore</a> was being used.</p>
+
+<h2 id="OakathonNovember2017-Handlingrequestsforblobsthataren'timmediatelyavailable">Handling requests for blobs that aren't immediately available</h2>
+<p>The prime example for this scenario is using AWS Glacier as an Oak data store option. Glacier as a data store doesn't make a lot of sense by itself but if used in the context of <a class="createlink" href="/confluence/pages/createpage.action?spaceKey=JCR&title=CompositeDataStore&linkCreation=true&fromPageId=115513479">CompositeDataStore</a> with some support for tiering or prioritization in the data stores, Glacier as the lowest priority, it might make sense.</p>
+
+<p>The biggest challenge with using Glacier is that unarchiving blobs from Glacier is a time-consuming task. The standard expectation is 4+ hours; expedited extraction is possible but even in this case retrieval is on the order of minutes. This clearly means we would need some mechanism for conveying "I can get the requested blob; I don't have it now, but I will have it in the future." Once a blob is unarchived it must be retrieved within 24 hours or it will return to the archived state.</p>
+
+<p>A new component (referred to as a "curator" in this discussion) was suggested. The role of the curator would be to retrieve unarchived objects from Glacier to a higher-level storage tier for future access. As proposed it would also have the role of applying policy to move infrequently accessed objects to lower tiers, eventually to Glacier. Because the curator moves objects, it knows where they are and when they have moved so Oak continues to know the whereabouts. If they are moved outside of Oak, it becomes difficult for Oak to keep track of blobs and their locations which could result in the composite data store requesting a blob from delegates where it doesn't exist anymore.</p>
+
+<p>The following items were discussed:</p>
+<ul>
+ <li>Should Glacier restore be an administrative task instead of something that occurs as the result of a standard user request? Since Glacier restores are expensive, there is risk that spurious user requests for blobs could result in unnecessary restores. This would mean that the responsibility of unarchiving would reside outside of Oak, either at the application level or we could assume users simply do it via their own AWS console or something.</li>
+ <li>In the context of tiered storage in a composite data store, Glacier storage is perhaps not useful unless S3DataStore is also being used, so we can probably assume S3DataStore. In which case, we can make assumptions about unarchiving. For example, an AWS Lambda could be used to move code from one store to another.</li>
+ <li>How does garbage collection work across multiple stores? (Open issue)</li>
+ <li>Curator component probably belongs within the context of oak-blob-composite, not as a separate bundle.</li>
+ <li>S3 IA has had some very rudimentary testing with Oak and should be much simpler to use. Would this be sufficient to meet a user desire for lower-cost storage and thus minimize the need to take on the additional complexity of using Glacier?</li>
+ <li>How do we get the curator to avoid archiving things that don't want to be archived? There may be some blobs that we never want to archive. Some suggestions:
+ <ul>
+ <li>Size (some concerns with this; while it is easy to determine whether a blob exceeds the minimum size requirement, it is much harder to come up with a meaningful size. For example, blob thumbnails should probably never be archived for user experience purposes, but what is the correct size that would include every conceivable thumbnail but not exclude archiving things that we want to archive?)</li>
+ <li>Last accessed time</li>
+ <li>Some other items, like index segments, should never be archived</li>
+ <li>Could we only archive certain parts of the tree?</li>
+ </ul>
+ </li>
+ <li>The blob store deduplicates blobs, which means multiple nodes may refer to the same blob. So the curator would need to be aware of all nodes referring to a blob and only archive if all nodes agree it should be archived. One idea given was to use a similar pattern as for garbage collection, e.g. during a mark phase blobs can be marked as "don't move" if any node votes it shouldn't be moved to a lower priority tier.</li>
+ <li>Once something has been moved, instead of removing the reference at the higher tier the record could be replaced with a placeholder indicating where it moved to.
+<br class="atl-forced-newline"/></li>
+</ul>
+
+
+<p>Toward the end of the discussion, it was brought up that perhaps we could get by without a Glacier data store delegate for composite data store and still allow users to store things in Glacier. It would require customization outside of Oak which could either be an application or simple administrative tasks. In such a case, as an example "archiving" a blob would mean it doesn't get deleted from a higher tier blob store, but rather gets replaced with some sort of marker that the user can understand as "archived". The custom code would copy the blob to archive. Unarchiving would require external effort to revert the process. So in other words the "application layer" (which could also just be admin scripts) is responsible for moving blobs from one tier to the other, as well as keeping track of the state of blobs. Difficulty comes in determining which blob is the one that should be moved and knowing whether it can be moved, dealing with multiple ref
erences, etc. but it is certainly possible to be done outside of Oak.</p>
+
+<p>Action plan: MattR to explore the need further and determine if using cold storage options is a real requirement and if doing it in Oak is really needed.</p>
+ </div>
+
+
+
+ </div> </div>
+ <div id="footer" role="contentinfo">
+ <section class="footer-body">
+ <p>Document generated by Confluence on May 20, 2019 11:11</p>
+ <div id="footer-logo"><a href="http://www.atlassian.com/">Atlassian</a></div>
+ </section>
+ </div>
+ </div> </body>
+</html>
Propchange: jackrabbit/site/live/archive/wiki/JCR/Oakathon-November-2017_115513479.html
------------------------------------------------------------------------------
svn:eol-style = native
Propchange: jackrabbit/site/live/archive/wiki/JCR/Oakathon-November-2017_115513479.html
------------------------------------------------------------------------------
svn:executable = *