You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@jena.apache.org by Andy Seaborne <an...@seaborne.org> on 2022/01/02 16:23:05 UTC

Fuseki Evolution

A collection of thoughts and discussion points about Fuseki.

== Jena 4.4.0

There is a vue-based rewrite of the UI thanks to Bruno (yea!). The new
UI is in the codebase and the LICENSE and NOTICE files updated. There
are no blockers for releasing this in 4.4.0 release. Prior to the
kerfuffle over security, the hope was an earlier-then-usual 4.4.0
because the UI only just missed 4.3.x.

== javax-jakarta transition

jakarta is JavaEE at the Eclipse Foundation).

At some time, there will need to be transition from javax.* to jakarta.*
for the package imports of javax.* that relate to Java EE.

For Fuseki that is mainly for javax.servlet but also an implication for
the WAR file.

For the Fuseki code itself, there isn't much impact, just a rename.
Partially this is because Fuseki does not use JavaEE features. For
example, request dispatch is not done with web.xml (Fuseki dispatch is
dynamic, not a static setup). FusekiMain does not use web.xml at all.

Jetty version 11 is the same as Jetty version 10 except that "javax" ->
"jakarta" renaming. A big bang switch over but it should be mechanical
translation.

There are two dependencies that use javax.servlet -- commons-fileupload
and shiro.

commons-fileupload code does not update very often (last version was
2018). It is small and Fuseki only uses a small part of it so we could
adopt the code we need for file upload from HTML pages.

shiro is an unknown.

The WAR file is more of a problem - we have already had someone try to
run Fuseki in Tomcat 10 and it fails. All Tomcat 9 webapps fail in
Tomcat 10.

Tomcat10 is based on jakarta while Tomcat9 is javax.

Tomcat has a migration tool: tomcat-jakartaee-migration but otherwise
it's a incompatible change. (I haven't tried the tool).

I haven't come across the reverse conversion. Maybe maven-shade-plugin
will do it.

== Distributing the WAR file.

The apache-jena-fuseki zip/tar.gz download is getting quite big. It has
both the war file and the standalong Fuseki server (jena-fuseki-fulljar).

One option is to change to providing the WAR file by a link on the
project download page, and note in the apache-jena-fuseki README.

The link could be in /binaries or to maven central.

What is hard to determine is how important the war file version actually
is nowadays. Should we focus on a runnable server Fuseki or are
multi-webapp hosting containers still common for semantic web data?

Or change the WAR file to be less webapp internally? Have a simple "all
URLs" grabber in web.xml and feed it into a wrapped FusekiMain?

== Standalone server - switch to Fuseki Main+UI

Ideally, long term, the standalone server would switch to being
Fuseli/Main + UI + Admin module. At the moment it is Jetty+and the same
code as the war file version.

Re: Fuseki Evolution

Posted by Andy Seaborne <an...@apache.org>.


On 05/01/2022 11:49, LB wrote:
> Nice progress so far.
> 
> Just a probably dumb question: how do I start Fuseki with UI now when 
> enabling GeoSPARQL via assembler? I tried to use the fulljar package
> 
>> java -jar target/jena-fuseki-fulljar-4.4.0-SNAPSHOT.jar --desc 
>> ~/fuseki-geosparql.ttl /ds
> 
> but this seems to lack the webapp context somehow?
> 
>> [2022-01-05 12:48:20] Server     INFO Running in read-only mode for /ds
>> [2022-01-05 12:48:20] Server     ERROR Can't find resourceBase (tried 
>> webapp, target/webapp, /home/user/./webapp and 
>> /home/user/./target/webapp)
>> [2022-01-05 12:48:20] Server     ERROR Failed to start
> the webapp resources are inside the JAR file as far as I can see but it 
> doesn't find them?
> 
> 
> it works when running the JAR from the jena-fuseki-webapp folder though

Yes, I think that is the way. teh files aren't in the jar in development 
or in 4.3.2. Did it ever work on its own? its shipped as 
fuseki-server.jar in apache-jena-fuseki-VER.zip

Sometime (tm) FMod_UI will bundle it into a jar.

     Andy

> 
> 
> Lorenz
> 
> 
> On 03.01.22 13:11, Andy Seaborne wrote:
>> On 02/01/2022 23:28, Bruno P. Kinoshita wrote:
>>>   Hi Andy,
>>>
>>> Hooray on seeing the new UI. I'm updating the JS dependencies (things 
>>> move fast in JS!), and should have a PR ready for review soon-ish. 
>>> Only issue was a new version of Bootstrap, but shouldn't be a big 
>>> issue. I'll start adding unit tests, and later e2e tests, but this 
>>> shouldn't block a release, since manually testing Fuseki is not too 
>>> hard as it's a small app at the moment.
>>
>> Cool - I was going to ask about how to best keep dependencies up to date.
>>
>> It sounds like ncu does the work, maybe there is a dependabot setting 
>> but versioning is not as smoothly automatic as java has tended to be 
>> for us. java-wise, we have only pretty stable dependencies.
>>
>>>> == javax-jakarta transition> (...)
>>>> commons-fileupload code does not update very often (last version was
>>>> 2018). It is small and Fuseki only uses a small part of it so we could
>>>> adopt the code we need for file upload from HTML pages.
>>>
>>> If there's an easy fix for commons-fileupload I can try to work with 
>>> others to get it merged and then release it (I'm working on a 
>>> commons-imaging release ATM)
>>
>> The technical work is change the naming.
>>
>> But.
>>
>> It is an incompatible change and a lot of the world is still on 
>> javax.servlet.  My guess is that old-world is still the majority.
>> I haven't seen any way to have a mix of javax and jakarta. The global 
>> change over is going to be a very long tail.
>>
>> For Fuseki, there's no rush.
>>
>>> As for the WAR and standalone versions; I don't understand very well 
>>> all the packaging, and how users are running it. I think the Java 
>>> layer could be just standalone app, with assembly/maven-plugins to 
>>> create a separate WAR file to upload with the release if users 
>>> want/need it.
>>
>> That's my hope. Have the WAR file as a extra tweak. At the moment
>>
>> jena-fuseki-core
>>    -> jena-fuseki-weabpp (admin and was the web pages; webapp startup)
>>         -> jena-fuseki-war (packaging, no code) -- drop-in for Tomcat
>>         -> jena-fuseki-fulljar (packaging, no code)
>>    -> jena-fuseki-main (FusekiServer.Builder, CLI)
>>         -> jena-fuseki-server (packaging)
>>         ... modules ...
>>         -> jena-fuseki-geosparrql
>>
>> "fulljar" is adding Jetty as the webapp container running the webapp 
>> code. So logically, its the war file + runtime.
>>
>>> The Vue UI, at the moment, produces a single app. But we could 
>>> organize it in a similar way. Have the core code in a JS module, and 
>>> another JS module that consumes the core, and produces a complete UI 
>>> with admin + query/editor. Or we could have separate JS modules. One 
>>> for sparqler, one for admin, one for query/editor (just would need to 
>>> move the common components to the core module).
>>>
>>> But starting simple as we are doing is a lot easier.
>>
>> +1 !
>>
>> >
>> I think you might be the best one to tell how the UI must evolve :) 
>> I'm just happy to see the dependencies managed with npm/yarn. We might 
>> start seeing dependabot PR's for JS soon.
>>>
>>>   Here's an example of an application with multiple modules, which 
>>> uses TypeScript but we don't need to use TypeScript if we don't want 
>>> to: https://github.com/jupyterlab/jupyterlab/tree/master/packages. 
>>> They work similar to Maven modules, with the parent module as the 
>>> maven parent 
>>> https://github.com/jupyterlab/jupyterlab/blob/602b05399b0ca762613c8f560a49b15abdefee39/package.json#L6-L25. 
>>>
>>
>> Thanks for the references.
>>
>>     Andy
>>
>>>
>>>
>>> Thanks!
>>>
>>> -Bruno
>>>
>>>
>>>
>>>      On Monday, 3 January 2022, 05:23:11 am NZDT, Andy Seaborne 
>>> <an...@seaborne.org> wrote:
>>>     A collection of thoughts and discussion points about Fuseki.
>>>
>>> == Jena 4.4.0
>>>
>>> There is a vue-based rewrite of the UI thanks to Bruno (yea!). The new
>>> UI is in the codebase and the LICENSE and NOTICE files updated. There
>>> are no blockers for releasing this in 4.4.0 release. Prior to the
>>> kerfuffle over security, the hope was an earlier-then-usual 4.4.0
>>> because the UI only just missed 4.3.x.
>>>
>>> == javax-jakarta transition
>>>
>>> jakarta is JavaEE at the Eclipse Foundation).
>>>
>>> At some time, there will need to be transition from javax.* to jakarta.*
>>> for the package imports of javax.* that relate to Java EE.
>>>
>>> For Fuseki that is mainly for javax.servlet but also an implication for
>>> the WAR file.
>>>
>>> For the Fuseki code itself, there isn't much impact, just a rename.
>>> Partially this is because Fuseki does not use JavaEE features. For
>>> example, request dispatch is not done with web.xml (Fuseki dispatch is
>>> dynamic, not a static setup). FusekiMain does not use web.xml at all.
>>>
>>> Jetty version 11 is the same as Jetty version 10 except that "javax" ->
>>> "jakarta" renaming. A big bang switch over but it should be mechanical
>>> translation.
>>>
>>> There are two dependencies that use javax.servlet -- commons-fileupload
>>> and shiro.
>>>
>>> commons-fileupload code does not update very often (last version was
>>> 2018). It is small and Fuseki only uses a small part of it so we could
>>> adopt the code we need for file upload from HTML pages.
>>>
>>> shiro is an unknown.
>>>
>>> The WAR file is more of a problem - we have already had someone try to
>>> run Fuseki in Tomcat 10 and it fails. All Tomcat 9 webapps fail in
>>> Tomcat 10.
>>>
>>> Tomcat10 is based on jakarta while Tomcat9 is javax.
>>>
>>> Tomcat has a migration tool: tomcat-jakartaee-migration but otherwise
>>> it's a incompatible change. (I haven't tried the tool).
>>>
>>> I haven't come across the reverse conversion. Maybe maven-shade-plugin
>>> will do it.
>>>
>>> == Distributing the WAR file.
>>>
>>> The apache-jena-fuseki zip/tar.gz download is getting quite big. It has
>>> both the war file and the standalong Fuseki server 
>>> (jena-fuseki-fulljar).
>>>
>>> One option is to change to providing the WAR file by a link on the
>>> project download page, and note in the apache-jena-fuseki README.
>>>
>>> The link could be in /binaries or to maven central.
>>>
>>> What is hard to determine is how important the war file version actually
>>> is nowadays.  Should we focus on a runnable server Fuseki or are
>>> multi-webapp hosting containers still common for semantic web data?
>>>
>>> Or change the WAR file to be less webapp internally? Have a simple "all
>>> URLs" grabber in web.xml and feed it into a wrapped FusekiMain?
>>>
>>> == Standalone server - switch to Fuseki Main+UI
>>>
>>> Ideally, long term, the standalone server would switch to being
>>> Fuseli/Main + UI + Admin module. At the moment it is Jetty+and the same
>>> code as the war file version.
>>>
>>> More on Fuseki modules below.
>>>
>>> Fuseki/main and Fuseki/webapp differ in how they startup and whether the
>>> additional servlets like admin are routed by web.xml or configured by
>>> the HTTP server builder in java code setup directly into Jetty.
>>>
>>> == Modules
>>>
>>> A Fuseki Module is loadable code that gets called in the server
>>> lifecycle, principally getting called during server build with direct
>>> access into the Fuseki server datastructures. A module can add new
>>> features, modify the Fuseki server as it is being built or simply
>>> monitor operations.
>>>
>>> It only works with FusekiMain - FusekiMain has a clearly defined
>>> lifecycle. The webapp is a "build-once" but also tied to the fixed
>>> web.xml routing.
>>>
>>> https://github.com/apache/jena/blob/main/jena-fuseki2/jena-fuseki-main/src/main/java/org/apache/jena/fuseki/main/sys/FusekiModule.java 
>>>
>>>
>>> This has been "experimental" in 4.3.x. I have (not-production-ready)
>>> modules:
>>>
>>> FMod_UI: jar file serving the static content of the UI JS/CSS/HTML
>>> directly from the jar file. No fixed disk area for static content.
>>>
>>> FMod_Admin: makes the admin code work as a Fuseki module.
>>> This would also be a chance to simplify the design.
>>> Thius experimental module is compatible with the on-disk layout of
>>> FusekiWebapp admin.
>>>
>>> FMod_Shiro (sktech): looks possible. A Fuseki module can add servlet
>>> filters to the servlet dispatch chain.
>>>
>>> FMod_FusekiKafka: Adds Kafka topics as an data input channel to Fuseki.
>>> It can transport data, patches or SPARQL Update requests and routes them
>>> to a dataset. (£job related.)
>>>
>>> FMod_ProvidePATCH: Add handling of HTTP verb PATCH.
>>> This is in "ExFusekiMain_3_FusekiModule.java"
>>>
>>> FMod_GraphAccessCtl: Configure the graph level security
>>> (jena-fuseki-access).
>>>
>>> FMod_ABAC: (£job related). Triple level attribute-based security.
>>>
>>> And recently mentioned: FMod_GeoSPARQL as a way of delivering
>>> jena-fuseki-geosparql capability in a general FusekiMain server.
>>>
>>> Some of these generate a new requirement not currently supported - the
>>> ability to add arguments to the command line.
>>>
>>>      Andy
>>>

Re: Fuseki Evolution

Posted by LB <co...@googlemail.com.INVALID>.

Nice progress so far.

Just a probably dumb question: how do I start Fuseki with UI now when 
enabling GeoSPARQL via assembler? I tried to use the fulljar package

> java -jar target/jena-fuseki-fulljar-4.4.0-SNAPSHOT.jar --desc 
> ~/fuseki-geosparql.ttl /ds

but this seems to lack the webapp context somehow?

> [2022-01-05 12:48:20] Server     INFO Running in read-only mode for /ds
> [2022-01-05 12:48:20] Server     ERROR Can't find resourceBase (tried 
> webapp, target/webapp, /home/user/./webapp and 
> /home/user/./target/webapp)
> [2022-01-05 12:48:20] Server     ERROR Failed to start
the webapp resources are inside the JAR file as far as I can see but it 
doesn't find them?


it works when running the JAR from the jena-fuseki-webapp folder though


Lorenz


On 03.01.22 13:11, Andy Seaborne wrote:
> On 02/01/2022 23:28, Bruno P. Kinoshita wrote:
>>   Hi Andy,
>>
>> Hooray on seeing the new UI. I'm updating the JS dependencies (things 
>> move fast in JS!), and should have a PR ready for review soon-ish. 
>> Only issue was a new version of Bootstrap, but shouldn't be a big 
>> issue. I'll start adding unit tests, and later e2e tests, but this 
>> shouldn't block a release, since manually testing Fuseki is not too 
>> hard as it's a small app at the moment.
>
> Cool - I was going to ask about how to best keep dependencies up to date.
>
> It sounds like ncu does the work, maybe there is a dependabot setting 
> but versioning is not as smoothly automatic as java has tended to be 
> for us. java-wise, we have only pretty stable dependencies.
>
>>> == javax-jakarta transition> (...)
>>> commons-fileupload code does not update very often (last version was
>>> 2018). It is small and Fuseki only uses a small part of it so we could
>>> adopt the code we need for file upload from HTML pages.
>>
>> If there's an easy fix for commons-fileupload I can try to work with 
>> others to get it merged and then release it (I'm working on a 
>> commons-imaging release ATM)
>
> The technical work is change the naming.
>
> But.
>
> It is an incompatible change and a lot of the world is still on 
> javax.servlet.  My guess is that old-world is still the majority.
> I haven't seen any way to have a mix of javax and jakarta. The global 
> change over is going to be a very long tail.
>
> For Fuseki, there's no rush.
>
>> As for the WAR and standalone versions; I don't understand very well 
>> all the packaging, and how users are running it. I think the Java 
>> layer could be just standalone app, with assembly/maven-plugins to 
>> create a separate WAR file to upload with the release if users 
>> want/need it.
>
> That's my hope. Have the WAR file as a extra tweak. At the moment
>
> jena-fuseki-core
>    -> jena-fuseki-weabpp (admin and was the web pages; webapp startup)
>         -> jena-fuseki-war (packaging, no code) -- drop-in for Tomcat
>         -> jena-fuseki-fulljar (packaging, no code)
>    -> jena-fuseki-main (FusekiServer.Builder, CLI)
>         -> jena-fuseki-server (packaging)
>         ... modules ...
>         -> jena-fuseki-geosparrql
>
> "fulljar" is adding Jetty as the webapp container running the webapp 
> code. So logically, its the war file + runtime.
>
>> The Vue UI, at the moment, produces a single app. But we could 
>> organize it in a similar way. Have the core code in a JS module, and 
>> another JS module that consumes the core, and produces a complete UI 
>> with admin + query/editor. Or we could have separate JS modules. One 
>> for sparqler, one for admin, one for query/editor (just would need to 
>> move the common components to the core module).
>>
>> But starting simple as we are doing is a lot easier.
>
> +1 !
>
> >
> I think you might be the best one to tell how the UI must evolve :) 
> I'm just happy to see the dependencies managed with npm/yarn. We might 
> start seeing dependabot PR's for JS soon.
>>
>>   Here's an example of an application with multiple modules, which 
>> uses TypeScript but we don't need to use TypeScript if we don't want 
>> to: https://github.com/jupyterlab/jupyterlab/tree/master/packages. 
>> They work similar to Maven modules, with the parent module as the 
>> maven parent 
>> https://github.com/jupyterlab/jupyterlab/blob/602b05399b0ca762613c8f560a49b15abdefee39/package.json#L6-L25.
>
> Thanks for the references.
>
>     Andy
>
>>
>>
>> Thanks!
>>
>> -Bruno
>>
>>
>>
>>      On Monday, 3 January 2022, 05:23:11 am NZDT, Andy Seaborne 
>> <an...@seaborne.org> wrote:
>>     A collection of thoughts and discussion points about Fuseki.
>>
>> == Jena 4.4.0
>>
>> There is a vue-based rewrite of the UI thanks to Bruno (yea!). The new
>> UI is in the codebase and the LICENSE and NOTICE files updated. There
>> are no blockers for releasing this in 4.4.0 release. Prior to the
>> kerfuffle over security, the hope was an earlier-then-usual 4.4.0
>> because the UI only just missed 4.3.x.
>>
>> == javax-jakarta transition
>>
>> jakarta is JavaEE at the Eclipse Foundation).
>>
>> At some time, there will need to be transition from javax.* to jakarta.*
>> for the package imports of javax.* that relate to Java EE.
>>
>> For Fuseki that is mainly for javax.servlet but also an implication for
>> the WAR file.
>>
>> For the Fuseki code itself, there isn't much impact, just a rename.
>> Partially this is because Fuseki does not use JavaEE features. For
>> example, request dispatch is not done with web.xml (Fuseki dispatch is
>> dynamic, not a static setup). FusekiMain does not use web.xml at all.
>>
>> Jetty version 11 is the same as Jetty version 10 except that "javax" ->
>> "jakarta" renaming. A big bang switch over but it should be mechanical
>> translation.
>>
>> There are two dependencies that use javax.servlet -- commons-fileupload
>> and shiro.
>>
>> commons-fileupload code does not update very often (last version was
>> 2018). It is small and Fuseki only uses a small part of it so we could
>> adopt the code we need for file upload from HTML pages.
>>
>> shiro is an unknown.
>>
>> The WAR file is more of a problem - we have already had someone try to
>> run Fuseki in Tomcat 10 and it fails. All Tomcat 9 webapps fail in
>> Tomcat 10.
>>
>> Tomcat10 is based on jakarta while Tomcat9 is javax.
>>
>> Tomcat has a migration tool: tomcat-jakartaee-migration but otherwise
>> it's a incompatible change. (I haven't tried the tool).
>>
>> I haven't come across the reverse conversion. Maybe maven-shade-plugin
>> will do it.
>>
>> == Distributing the WAR file.
>>
>> The apache-jena-fuseki zip/tar.gz download is getting quite big. It has
>> both the war file and the standalong Fuseki server 
>> (jena-fuseki-fulljar).
>>
>> One option is to change to providing the WAR file by a link on the
>> project download page, and note in the apache-jena-fuseki README.
>>
>> The link could be in /binaries or to maven central.
>>
>> What is hard to determine is how important the war file version actually
>> is nowadays.  Should we focus on a runnable server Fuseki or are
>> multi-webapp hosting containers still common for semantic web data?
>>
>> Or change the WAR file to be less webapp internally? Have a simple "all
>> URLs" grabber in web.xml and feed it into a wrapped FusekiMain?
>>
>> == Standalone server - switch to Fuseki Main+UI
>>
>> Ideally, long term, the standalone server would switch to being
>> Fuseli/Main + UI + Admin module. At the moment it is Jetty+and the same
>> code as the war file version.
>>
>> More on Fuseki modules below.
>>
>> Fuseki/main and Fuseki/webapp differ in how they startup and whether the
>> additional servlets like admin are routed by web.xml or configured by
>> the HTTP server builder in java code setup directly into Jetty.
>>
>> == Modules
>>
>> A Fuseki Module is loadable code that gets called in the server
>> lifecycle, principally getting called during server build with direct
>> access into the Fuseki server datastructures. A module can add new
>> features, modify the Fuseki server as it is being built or simply
>> monitor operations.
>>
>> It only works with FusekiMain - FusekiMain has a clearly defined
>> lifecycle. The webapp is a "build-once" but also tied to the fixed
>> web.xml routing.
>>
>> https://github.com/apache/jena/blob/main/jena-fuseki2/jena-fuseki-main/src/main/java/org/apache/jena/fuseki/main/sys/FusekiModule.java 
>>
>>
>> This has been "experimental" in 4.3.x. I have (not-production-ready)
>> modules:
>>
>> FMod_UI: jar file serving the static content of the UI JS/CSS/HTML
>> directly from the jar file. No fixed disk area for static content.
>>
>> FMod_Admin: makes the admin code work as a Fuseki module.
>> This would also be a chance to simplify the design.
>> Thius experimental module is compatible with the on-disk layout of
>> FusekiWebapp admin.
>>
>> FMod_Shiro (sktech): looks possible. A Fuseki module can add servlet
>> filters to the servlet dispatch chain.
>>
>> FMod_FusekiKafka: Adds Kafka topics as an data input channel to Fuseki.
>> It can transport data, patches or SPARQL Update requests and routes them
>> to a dataset. (£job related.)
>>
>> FMod_ProvidePATCH: Add handling of HTTP verb PATCH.
>> This is in "ExFusekiMain_3_FusekiModule.java"
>>
>> FMod_GraphAccessCtl: Configure the graph level security
>> (jena-fuseki-access).
>>
>> FMod_ABAC: (£job related). Triple level attribute-based security.
>>
>> And recently mentioned: FMod_GeoSPARQL as a way of delivering
>> jena-fuseki-geosparql capability in a general FusekiMain server.
>>
>> Some of these generate a new requirement not currently supported - the
>> ability to add arguments to the command line.
>>
>>      Andy
>>

Re: Fuseki Evolution

Posted by Andy Seaborne <an...@apache.org>.

On 02/01/2022 23:28, Bruno P. Kinoshita wrote:
>   Hi Andy,
> 
> Hooray on seeing the new UI. I'm updating the JS dependencies (things move fast in JS!), and should have a PR ready for review soon-ish. Only issue was a new version of Bootstrap, but shouldn't be a big issue. I'll start adding unit tests, and later e2e tests, but this shouldn't block a release, since manually testing Fuseki is not too hard as it's a small app at the moment.

Cool - I was going to ask about how to best keep dependencies up to date.

It sounds like ncu does the work, maybe there is a dependabot setting 
but versioning is not as smoothly automatic as java has tended to be for 
us. java-wise, we have only pretty stable dependencies.

>> == javax-jakarta transition> (...)
>> commons-fileupload code does not update very often (last version was
>> 2018). It is small and Fuseki only uses a small part of it so we could
>> adopt the code we need for file upload from HTML pages.
> 
> If there's an easy fix for commons-fileupload I can try to work with others to get it merged and then release it (I'm working on a commons-imaging release ATM)

The technical work is change the naming.

But.

It is an incompatible change and a lot of the world is still on 
javax.servlet.  My guess is that old-world is still the majority.
I haven't seen any way to have a mix of javax and jakarta. The global 
change over is going to be a very long tail.

For Fuseki, there's no rush.

> As for the WAR and standalone versions; I don't understand very well all the packaging, and how users are running it. I think the Java layer could be just standalone app, with assembly/maven-plugins to create a separate WAR file to upload with the release if users want/need it.

That's my hope. Have the WAR file as a extra tweak. At the moment

jena-fuseki-core
    -> jena-fuseki-weabpp (admin and was the web pages; webapp startup)
         -> jena-fuseki-war (packaging, no code) -- drop-in for Tomcat
         -> jena-fuseki-fulljar (packaging, no code)
    -> jena-fuseki-main (FusekiServer.Builder, CLI)
         -> jena-fuseki-server (packaging)
         ... modules ...
         -> jena-fuseki-geosparrql

"fulljar" is adding Jetty as the webapp container running the webapp 
code. So logically, its the war file + runtime.

> The Vue UI, at the moment, produces a single app. But we could organize it in a similar way. Have the core code in a JS module, and another JS module that consumes the core, and produces a complete UI with admin + query/editor. Or we could have separate JS modules. One for sparqler, one for admin, one for query/editor (just would need to move the common components to the core module).
> 
> But starting simple as we are doing is a lot easier.

+1 !

 >
I think you might be the best one to tell how the UI must evolve :) I'm 
just happy to see the dependencies managed with npm/yarn. We might start 
seeing dependabot PR's for JS soon.
> 
>   Here's an example of an application with multiple modules, which uses TypeScript but we don't need to use TypeScript if we don't want to: https://github.com/jupyterlab/jupyterlab/tree/master/packages. They work similar to Maven modules, with the parent module as the maven parent https://github.com/jupyterlab/jupyterlab/blob/602b05399b0ca762613c8f560a49b15abdefee39/package.json#L6-L25.

Thanks for the references.

     Andy

> 
> 
> Thanks!
> 
> -Bruno
> 
> 
> 
>      On Monday, 3 January 2022, 05:23:11 am NZDT, Andy Seaborne <an...@seaborne.org> wrote:
>   
>   A collection of thoughts and discussion points about Fuseki.
> 
> == Jena 4.4.0
> 
> There is a vue-based rewrite of the UI thanks to Bruno (yea!). The new
> UI is in the codebase and the LICENSE and NOTICE files updated. There
> are no blockers for releasing this in 4.4.0 release. Prior to the
> kerfuffle over security, the hope was an earlier-then-usual 4.4.0
> because the UI only just missed 4.3.x.
> 
> == javax-jakarta transition
> 
> jakarta is JavaEE at the Eclipse Foundation).
> 
> At some time, there will need to be transition from javax.* to jakarta.*
> for the package imports of javax.* that relate to Java EE.
> 
> For Fuseki that is mainly for javax.servlet but also an implication for
> the WAR file.
> 
> For the Fuseki code itself, there isn't much impact, just a rename.
> Partially this is because Fuseki does not use JavaEE features. For
> example, request dispatch is not done with web.xml (Fuseki dispatch is
> dynamic, not a static setup). FusekiMain does not use web.xml at all.
> 
> Jetty version 11 is the same as Jetty version 10 except that "javax" ->
> "jakarta" renaming. A big bang switch over but it should be mechanical
> translation.
> 
> There are two dependencies that use javax.servlet -- commons-fileupload
> and shiro.
> 
> commons-fileupload code does not update very often (last version was
> 2018). It is small and Fuseki only uses a small part of it so we could
> adopt the code we need for file upload from HTML pages.
> 
> shiro is an unknown.
> 
> The WAR file is more of a problem - we have already had someone try to
> run Fuseki in Tomcat 10 and it fails. All Tomcat 9 webapps fail in
> Tomcat 10.
> 
> Tomcat10 is based on jakarta while Tomcat9 is javax.
> 
> Tomcat has a migration tool: tomcat-jakartaee-migration but otherwise
> it's a incompatible change. (I haven't tried the tool).
> 
> I haven't come across the reverse conversion. Maybe maven-shade-plugin
> will do it.
> 
> == Distributing the WAR file.
> 
> The apache-jena-fuseki zip/tar.gz download is getting quite big. It has
> both the war file and the standalong Fuseki server (jena-fuseki-fulljar).
> 
> One option is to change to providing the WAR file by a link on the
> project download page, and note in the apache-jena-fuseki README.
> 
> The link could be in /binaries or to maven central.
> 
> What is hard to determine is how important the war file version actually
> is nowadays.  Should we focus on a runnable server Fuseki or are
> multi-webapp hosting containers still common for semantic web data?
> 
> Or change the WAR file to be less webapp internally? Have a simple "all
> URLs" grabber in web.xml and feed it into a wrapped FusekiMain?
> 
> == Standalone server - switch to Fuseki Main+UI
> 
> Ideally, long term, the standalone server would switch to being
> Fuseli/Main + UI + Admin module. At the moment it is Jetty+and the same
> code as the war file version.
> 
> More on Fuseki modules below.
> 
> Fuseki/main and Fuseki/webapp differ in how they startup and whether the
> additional servlets like admin are routed by web.xml or configured by
> the HTTP server builder in java code setup directly into Jetty.
> 
> == Modules
> 
> A Fuseki Module is loadable code that gets called in the server
> lifecycle, principally getting called during server build with direct
> access into the Fuseki server datastructures. A module can add new
> features, modify the Fuseki server as it is being built or simply
> monitor operations.
> 
> It only works with FusekiMain - FusekiMain has a clearly defined
> lifecycle. The webapp is a "build-once" but also tied to the fixed
> web.xml routing.
> 
> https://github.com/apache/jena/blob/main/jena-fuseki2/jena-fuseki-main/src/main/java/org/apache/jena/fuseki/main/sys/FusekiModule.java
> 
> This has been "experimental" in 4.3.x. I have (not-production-ready)
> modules:
> 
> FMod_UI: jar file serving the static content of the UI JS/CSS/HTML
> directly from the jar file. No fixed disk area for static content.
> 
> FMod_Admin: makes the admin code work as a Fuseki module.
> This would also be a chance to simplify the design.
> Thius experimental module is compatible with the on-disk layout of
> FusekiWebapp admin.
> 
> FMod_Shiro (sktech): looks possible. A Fuseki module can add servlet
> filters to the servlet dispatch chain.
> 
> FMod_FusekiKafka: Adds Kafka topics as an data input channel to Fuseki.
> It can transport data, patches or SPARQL Update requests and routes them
> to a dataset. (£job related.)
> 
> FMod_ProvidePATCH: Add handling of HTTP verb PATCH.
> This is in "ExFusekiMain_3_FusekiModule.java"
> 
> FMod_GraphAccessCtl: Configure the graph level security
> (jena-fuseki-access).
> 
> FMod_ABAC: (£job related). Triple level attribute-based security.
> 
> And recently mentioned: FMod_GeoSPARQL as a way of delivering
> jena-fuseki-geosparql capability in a general FusekiMain server.
> 
> Some of these generate a new requirement not currently supported - the
> ability to add arguments to the command line.
> 
>      Andy
>    
>

Re: Fuseki Evolution

Posted by "Bruno P. Kinoshita" <br...@yahoo.com.br.INVALID>.

Hi Andy,

Hooray on seeing the new UI. I'm updating the JS dependencies (things move fast in JS!), and should have a PR ready for review soon-ish. Only issue was a new version of Bootstrap, but shouldn't be a big issue. I'll start adding unit tests, and later e2e tests, but this shouldn't block a release, since manually testing Fuseki is not too hard as it's a small app at the moment.

>== javax-jakarta transition> (...)
>commons-fileupload code does not update very often (last version was
>2018). It is small and Fuseki only uses a small part of it so we could
>adopt the code we need for file upload from HTML pages.

If there's an easy fix for commons-fileupload I can try to work with others to get it merged and then release it (I'm working on a commons-imaging release ATM)

As for the WAR and standalone versions; I don't understand very well all the packaging, and how users are running it. I think the Java layer could be just standalone app, with assembly/maven-plugins to create a separate WAR file to upload with the release if users want/need it.

The Vue UI, at the moment, produces a single app. But we could organize it in a similar way. Have the core code in a JS module, and another JS module that consumes the core, and produces a complete UI with admin + query/editor. Or we could have separate JS modules. One for sparqler, one for admin, one for query/editor (just would need to move the common components to the core module).

But starting simple as we are doing is a lot easier. I think you might be the best one to tell how the UI must evolve :) I'm just happy to see the dependencies managed with npm/yarn. We might start seeing dependabot PR's for JS soon.

Here's an example of an application with multiple modules, which uses TypeScript but we don't need to use TypeScript if we don't want to: https://github.com/jupyterlab/jupyterlab/tree/master/packages. They work similar to Maven modules, with the parent module as the maven parent https://github.com/jupyterlab/jupyterlab/blob/602b05399b0ca762613c8f560a49b15abdefee39/package.json#L6-L25.

Thanks!

-Bruno

On Monday, 3 January 2022, 05:23:11 am NZDT, Andy Seaborne <an...@seaborne.org> wrote:

A collection of thoughts and discussion points about Fuseki.

== Jena 4.4.0

== javax-jakarta transition

jakarta is JavaEE at the Eclipse Foundation).

At some time, there will need to be transition from javax.* to jakarta.*
for the package imports of javax.* that relate to Java EE.

For Fuseki that is mainly for javax.servlet but also an implication for
the WAR file.

Jetty version 11 is the same as Jetty version 10 except that "javax" ->
"jakarta" renaming. A big bang switch over but it should be mechanical
translation.

There are two dependencies that use javax.servlet -- commons-fileupload
and shiro.

commons-fileupload code does not update very often (last version was
2018). It is small and Fuseki only uses a small part of it so we could
adopt the code we need for file upload from HTML pages.

shiro is an unknown.

The WAR file is more of a problem - we have already had someone try to
run Fuseki in Tomcat 10 and it fails. All Tomcat 9 webapps fail in
Tomcat 10.

Tomcat10 is based on jakarta while Tomcat9 is javax.

Tomcat has a migration tool: tomcat-jakartaee-migration but otherwise
it's a incompatible change. (I haven't tried the tool).

I haven't come across the reverse conversion. Maybe maven-shade-plugin
will do it.

== Distributing the WAR file.

The apache-jena-fuseki zip/tar.gz download is getting quite big. It has
both the war file and the standalong Fuseki server (jena-fuseki-fulljar).

One option is to change to providing the WAR file by a link on the
project download page, and note in the apache-jena-fuseki README.

The link could be in /binaries or to maven central.

Or change the WAR file to be less webapp internally? Have a simple "all
URLs" grabber in web.xml and feed it into a wrapped FusekiMain?

== Standalone server - switch to Fuseki Main+UI

Ideally, long term, the standalone server would switch to being
Fuseli/Main + UI + Admin module. At the moment it is Jetty+and the same
code as the war file version.

Re: Fuseki Evolution

Posted by Andy Seaborne <an...@apache.org>.


On 02/01/2022 16:23, Andy Seaborne wrote:

> == javax-jakarta transition
> 
> jakarta is JavaEE at the Eclipse Foundation).
> 
> At some time, there will need to be transition from javax.* to jakarta.* 
> for the package imports of javax.* that relate to Java EE.
> 
> For Fuseki that is mainly for javax.servlet but also an implication for 
> the WAR file.
> 
> For the Fuseki code itself, there isn't much impact, just a rename. 
> Partially this is because Fuseki does not use JavaEE features. For 
> example, request dispatch is not done with web.xml (Fuseki dispatch is 
> dynamic, not a static setup). FusekiMain does not use web.xml at all.
> 
> Jetty version 11 is the same as Jetty version 10 except that "javax" -> 
> "jakarta" renaming. A big bang switch over but it should be mechanical 
> translation.
> 
> There are two dependencies that use javax.servlet -- commons-fileupload 
> and shiro.
> 
> commons-fileupload code does not update very often (last version was 
> 2018). It is small and Fuseki only uses a small part of it so we could 
> adopt the code we need for file upload from HTML pages.

There is a commons-fileupload2 in the pipeline.

This is for jakarta compatible.

https://issues.apache.org/jira/browse/FILEUPLOAD-309

     Andy

Re: Fuseki Evolution

Posted by Martynas Jusevičius <ma...@atomgraph.com>.

Hi,

From our side, the only distribution of Fuseki we are using is the
Docker image (plain server without UI).

On Sun, Jan 2, 2022 at 5:23 PM Andy Seaborne <an...@seaborne.org> wrote:
>
> A collection of thoughts and discussion points about Fuseki.
>
> == Jena 4.4.0
>
> There is a vue-based rewrite of the UI thanks to Bruno (yea!). The new
> UI is in the codebase and the LICENSE and NOTICE files updated. There
> are no blockers for releasing this in 4.4.0 release. Prior to the
> kerfuffle over security, the hope was an earlier-then-usual 4.4.0
> because the UI only just missed 4.3.x.
>
> == javax-jakarta transition
>
> jakarta is JavaEE at the Eclipse Foundation).
>
> At some time, there will need to be transition from javax.* to jakarta.*
> for the package imports of javax.* that relate to Java EE.
>
> For Fuseki that is mainly for javax.servlet but also an implication for
> the WAR file.
>
> For the Fuseki code itself, there isn't much impact, just a rename.
> Partially this is because Fuseki does not use JavaEE features. For
> example, request dispatch is not done with web.xml (Fuseki dispatch is
> dynamic, not a static setup). FusekiMain does not use web.xml at all.
>
> Jetty version 11 is the same as Jetty version 10 except that "javax" ->
> "jakarta" renaming. A big bang switch over but it should be mechanical
> translation.
>
> There are two dependencies that use javax.servlet -- commons-fileupload
> and shiro.
>
> commons-fileupload code does not update very often (last version was
> 2018). It is small and Fuseki only uses a small part of it so we could
> adopt the code we need for file upload from HTML pages.
>
> shiro is an unknown.
>
> The WAR file is more of a problem - we have already had someone try to
> run Fuseki in Tomcat 10 and it fails. All Tomcat 9 webapps fail in
> Tomcat 10.
>
> Tomcat10 is based on jakarta while Tomcat9 is javax.
>
> Tomcat has a migration tool: tomcat-jakartaee-migration but otherwise
> it's a incompatible change. (I haven't tried the tool).
>
> I haven't come across the reverse conversion. Maybe maven-shade-plugin
> will do it.
>
> == Distributing the WAR file.
>
> The apache-jena-fuseki zip/tar.gz download is getting quite big. It has
> both the war file and the standalong Fuseki server (jena-fuseki-fulljar).
>
> One option is to change to providing the WAR file by a link on the
> project download page, and note in the apache-jena-fuseki README.
>
> The link could be in /binaries or to maven central.
>
> What is hard to determine is how important the war file version actually
> is nowadays.  Should we focus on a runnable server Fuseki or are
> multi-webapp hosting containers still common for semantic web data?
>
> Or change the WAR file to be less webapp internally? Have a simple "all
> URLs" grabber in web.xml and feed it into a wrapped FusekiMain?
>
> == Standalone server - switch to Fuseki Main+UI
>
> Ideally, long term, the standalone server would switch to being
> Fuseli/Main + UI + Admin module. At the moment it is Jetty+and the same
> code as the war file version.
>
> More on Fuseki modules below.
>
> Fuseki/main and Fuseki/webapp differ in how they startup and whether the
> additional servlets like admin are routed by web.xml or configured by
> the HTTP server builder in java code setup directly into Jetty.
>
> == Modules
>
> A Fuseki Module is loadable code that gets called in the server
> lifecycle, principally getting called during server build with direct
> access into the Fuseki server datastructures. A module can add new
> features, modify the Fuseki server as it is being built or simply
> monitor operations.
>
> It only works with FusekiMain - FusekiMain has a clearly defined
> lifecycle. The webapp is a "build-once" but also tied to the fixed
> web.xml routing.
>
> https://github.com/apache/jena/blob/main/jena-fuseki2/jena-fuseki-main/src/main/java/org/apache/jena/fuseki/main/sys/FusekiModule.java
>
> This has been "experimental" in 4.3.x. I have (not-production-ready)
> modules:
>
> FMod_UI: jar file serving the static content of the UI JS/CSS/HTML
> directly from the jar file. No fixed disk area for static content.
>
> FMod_Admin: makes the admin code work as a Fuseki module.
> This would also be a chance to simplify the design.
> Thius experimental module is compatible with the on-disk layout of
> FusekiWebapp admin.
>
> FMod_Shiro (sktech): looks possible. A Fuseki module can add servlet
> filters to the servlet dispatch chain.
>
> FMod_FusekiKafka: Adds Kafka topics as an data input channel to Fuseki.
> It can transport data, patches or SPARQL Update requests and routes them
> to a dataset. (£job related.)
>
> FMod_ProvidePATCH: Add handling of HTTP verb PATCH.
> This is in "ExFusekiMain_3_FusekiModule.java"
>
> FMod_GraphAccessCtl: Configure the graph level security
> (jena-fuseki-access).
>
> FMod_ABAC: (£job related). Triple level attribute-based security.
>
> And recently mentioned: FMod_GeoSPARQL as a way of delivering
> jena-fuseki-geosparql capability in a general FusekiMain server.
>
> Some of these generate a new requirement not currently supported - the
> ability to add arguments to the command line.
>
>      Andy