You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by Tom Browder <to...@gmail.com> on 2020/10/03 18:08:06 UTC
[users@httpd] Re: Alternatives to SSI (server side includes)?
On Sat, Oct 3, 2020 at 12:18 Tom Browder <to...@gmail.com> wrote:
> I have been using server side includes since I started my websites on Apache
...
> Any suggestions for SSI replacement with a more asynchronous method?
Let me be more specific about the data flow I'm using with the landing
(home) page of my websites:
+ the first SSI line executes a CGI program that extracts the CGI and
SSL variables and their values. Data of interest include: requesting
IP, email address in the SSL client certificate, time of page load
+ the second line executes a CGI program that generates a link to a
page that presents the current and past statistics of the visitor
based on the stats in the db updated with the first CGI program
-Tom
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
RE: [users@httpd] Re: Alternatives to SSI (server side includes)?
[EXT]
Posted by James Smith <js...@sanger.ac.uk>.
Definitely SQLite will be a bottle neck in this system – not great for writing to – both Pg or MySQL would be an almost certainly better solution for repeated writing to.
You could get some simple gains by splitting the database up so that there is a database per site rather than a database for all 10 sites – unless there is data to be shared between them – as you would have less database lock contention on the individual databases. SQLlite is not designed to be used in a situation where there are lots of write locks. And does not scale very well as the database gets larger…
Also, just trying to get round what each visitor triggers in the CGI – and why it is a lot of work for the db – this is where you would need to make things easier for it & yourself, is the information you show useful to the user or could this just be processed some other way!
From: Tom Browder <to...@gmail.com>
Sent: 03 October 2020 20:29
To: users@httpd.apache.org
Subject: Re: [users@httpd] Re: Alternatives to SSI (server side includes)? [EXT]
On Sat, Oct 3, 2020 at 13:46 Scott A. Wozny <sa...@hotmail.com>> wrote:
Sounds like a job for AJAX, but before throwing out the baby with the bath water I'd seriously consider turning up logging with timestamps on your existing CGI and
That's a good idea, Scott, I've just been too lazy and debugging CGI is such a pain.
The clients are mostly casual browsers, but every visitor triggers the CGI so that's a lot of work for the db. Regarding the db it's currently an SQLite instance but I'm running about 10+ virtual sites off the same db and needing writes so that alone may be part of the problem.
Also, I do have a good Pg db running I've intended to use but just haven't gotten a "round tuit" yet.
But, could a web socket setup help do you think?
Best,
-Tom
--
The Wellcome Sanger Institute is operated by Genome Research
Limited, a charity registered in England with number 1021457 and a
company registered in England with number 2742969, whose registered
office is 215 Euston Road, London, NW1 2BE.
RE: [users@httpd] Re: Alternatives to SSI (server side includes)?
[EXT]
Posted by James Smith <js...@sanger.ac.uk>.
It’s probably the size of your SQLlite database – so I would look at reducing the size of that.
Still unclear what you are doing to know what the delay is – perhaps some sample code would be useful for us to look at – so we know what you parse & store; and also what you display if you display anything.
Another way of solving some of these issues is to use tell-tales – small images {blank gif/png} that are embedded into the HTML that are generated dynamically – this is the way google analytics/matamo etc do this sort of logging. Or just use AJAX.
James
From: Tom Browder <to...@gmail.com>
Sent: 05 October 2020 12:16
To: users@httpd.apache.org
Subject: Re: [users@httpd] Re: Alternatives to SSI (server side includes)? [EXT]
On Sun, Oct 4, 2020 at 13:05 Scott A. Wozny <sa...@hotmail.com>> wrote:
IMHO, Web Sockets aren't going to get you any real benefit here. The primary
Thanks, Scott. I do intend to look into the timing.
BTW, this website takes over eight seconds to load, and it uses the same CGI setup as my other sites:
https://psrr.info [psrr.info]<https://urldefense.proofpoint.com/v2/url?u=https-3A__psrr.info&d=DwMFaQ&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=oH2yp0ge1ecj4oDX0XM7vQ&m=ryAWP_lPitz08Sx1FEJB_b1M21KERCZi3Py2Ctc7jnA&s=U9695vfG7oBlMx_zaAviIxMBDu_L5y3crE5PaFZ5-io&e=>
Cheers!
-Tom
--
The Wellcome Sanger Institute is operated by Genome Research
Limited, a charity registered in England with number 1021457 and a
company registered in England with number 2742969, whose registered
office is 215 Euston Road, London, NW1 2BE.
Re: [users@httpd] Re: Alternatives to SSI (server side includes)?
Posted by Tom Browder <to...@gmail.com>.
On Sun, Oct 4, 2020 at 13:05 Scott A. Wozny <sa...@hotmail.com> wrote:
> IMHO, Web Sockets aren't going to get you any real benefit here. The
> primary
>
Thanks, Scott. I do intend to look into the timing.
BTW, this website takes over eight seconds to load, and it uses the same
CGI setup as my other sites:
https://psrr.info
Cheers!
-Tom
Re: [users@httpd] Re: Alternatives to SSI (server side includes)?
Posted by "Scott A. Wozny" <sa...@hotmail.com>.
IMHO, Web Sockets aren't going to get you any real benefit here. The primary point of web sockets is to keep a bidirectional conversation open for the long term during a visitor's session which makes it super beneficial to something like an infinite scroll, but it sounds like your problem is a one time rendered asset that's taking an inordinately long time to return. That's why I suggested AJAX but the other suggestions I've seen of iframes are also applicable to what you've given us about your scenario (but without seeing your page, iframes may not be a very 'pretty' answer).
When I'm facing an architectural efficiency question like this, I find it helpful to remember that the web is a document platform we've mutated into an application platform by sheer force of will. All the server cares about is returning each document in isolation. Everything else that makes it LOOK like a web page is responding to the client is JavaScript and CSS voodoo.
In your case it sounds like the limiting factor is how long that individual asset takes to return. Absent a rewrite of the inefficient asset, your primary goal is to take that object out of the critical path for the server to complete the main page return faster. In this case without you ALSO adding a way to make the slow asset a separate document return and once you've done that (AJAX, iframes, etc...), web sockets didn't buy you anything.
However, like all things related to technology, there's a lot of ways to accomplish any task. I still stand by my original suggestion that you at least look at the timings on your CGI before considering a total rewrite, though. 🙂
HTH,
Scott
P.S. Interesting debate on SO about web sockets vs AJAX that evolved over several years. 🙂 https://stackoverflow.com/questions/10377384/why-use-ajax-when-websockets-is-available
________________________________
From: Tom Browder <to...@gmail.com>
Sent: October 3, 2020 3:29 PM
To: users@httpd.apache.org <us...@httpd.apache.org>
Subject: Re: [users@httpd] Re: Alternatives to SSI (server side includes)?
On Sat, Oct 3, 2020 at 13:46 Scott A. Wozny <sa...@hotmail.com>> wrote:
Sounds like a job for AJAX, but before throwing out the baby with the bath water I'd seriously consider turning up logging with timestamps on your existing CGI and
That's a good idea, Scott, I've just been too lazy and debugging CGI is such a pain.
The clients are mostly casual browsers, but every visitor triggers the CGI so that's a lot of work for the db. Regarding the db it's currently an SQLite instance but I'm running about 10+ virtual sites off the same db and needing writes so that alone may be part of the problem.
Also, I do have a good Pg db running I've intended to use but just haven't gotten a "round tuit" yet.
But, could a web socket setup help do you think?
Best,
-Tom
Re: [users@httpd] Re: Alternatives to SSI (server side includes)?
Posted by Tom Browder <to...@gmail.com>.
On Sat, Oct 3, 2020 at 13:46 Scott A. Wozny <sa...@hotmail.com> wrote:
> Sounds like a job for AJAX, but before throwing out the baby with the bath
> water I'd seriously consider turning up logging with timestamps on your
> existing CGI and
>
That's a good idea, Scott, I've just been too lazy and debugging CGI is
such a pain.
The clients are mostly casual browsers, but every visitor triggers the CGI
so that's a lot of work for the db. Regarding the db it's currently an
SQLite instance but I'm running about 10+ virtual sites off the same db and
needing writes so that alone may be part of the problem.
Also, I do have a good Pg db running I've intended to use but just haven't
gotten a "round tuit" yet.
But, could a web socket setup help do you think?
Best,
-Tom
RE: [users@httpd] Re: Alternatives to SSI (server side includes)?
[EXT]
Posted by James Smith <js...@sanger.ac.uk>.
I frames have their use – but usually to include content from another site (e.g. google maps, you tube etc) – or to embed dynamic content that either needs to be dynamically updated and can’t do this with AJAX or you are struggling with CSS clashes as the iframe is a different document. Not sure if it would really help here – I think I would look at AJAX first – depends on what you are trying to do.
There are alternative ways you can do this – but depends on your server architecture – if you are using mod_perl for instances you can look at an output filter to add the data, if you are using PSGI you may be able to wrap additional layers around your code {won’t gain you much but would avoid additional calls to cgi scripts} and add the content into the page after the page has been rendered. With mod_perl you can do useful stuff with pnotes, in other cases you may be able to use environment variables if you are running in the same process {not easy if you are doing two separate calls}
The system we use is set up to do two stage caching – one which caches the content of the page with placeholders which then get processed with additional variables.
From: Tom Browder <to...@gmail.com>
Sent: 04 October 2020 11:44
To: users@httpd.apache.org
Subject: Re: [users@httpd] Re: Alternatives to SSI (server side includes)? [EXT]
On Sun, Oct 4, 2020 at 04:38 Rob De Langhe <ro...@twistfare.be>> wrote:
I simply use (or dynamically construct) a page with iframes, in which each iframe gets loaded by a separate CGI results;
Hm, I've always thought that iframes were frowned upon in modern practice. I'll have to read up on them
Thanks, Rob.
Cheers,
-Tom
--
The Wellcome Sanger Institute is operated by Genome Research
Limited, a charity registered in England with number 1021457 and a
company registered in England with number 2742969, whose registered
office is 215 Euston Road, London, NW1 2BE.
Re: [users@httpd] Re: Alternatives to SSI (server side includes)?
Posted by Tom Browder <to...@gmail.com>.
On Sun, Oct 4, 2020 at 04:38 Rob De Langhe <ro...@twistfare.be>
wrote:
> I simply use (or dynamically construct) a page with iframes, in which each
> iframe gets loaded by a separate CGI results;
>
Hm, I've always thought that iframes were frowned upon in modern practice.
I'll have to read up on them
Thanks, Rob.
Cheers,
-Tom
Re: [users@httpd] Re: Alternatives to SSI (server side includes)?
Posted by Rob De Langhe <ro...@twistfare.be>.
hi Tom,
I simply use (or dynamically construct) a page with iframes, in which
each iframe gets loaded by a separate CGI results;
this is the async way. You can trigger the loading of certain iframes
when user clicks on some button in the main part of the page, then a
Javascript function is called by that button to re-load the contents
of some iframe by a corresponding CGI.
No AJAX needed, but plain HTML + Javascript.
cheers
Rob
Quoting "Scott A. Wozny" <sa...@hotmail.com>:
> Sounds like a job for AJAX, but before throwing out the baby with
> the bath water I'd seriously consider turning up logging with
> timestamps on your existing CGI and seeing if SSI is a loser in its
> entirety or if it's one specific part of the process that is tanking
> your page load times that you can easily fix (you don't say much
> about that data store, but if you've crossed a tipping in regards to
> size or number of records, you may need to reconsider your data
> storage strategy).
>
> Also, if you've got really old / dumb clients AJAX might not be a
> great answer whereas SSI is always entirely under your control.
>
> Best of luck,
>
> Scott
>
>
> -------------------------
> FROM: Tom Browder <to...@gmail.com>
> SENT: October 3, 2020 2:08 PM
> TO: users@httpd.apache.org <us...@httpd.apache.org>
> SUBJECT: [users@httpd] Re: Alternatives to SSI (server side
> includes)?
>
> On Sat, Oct 3, 2020 at 12:18 Tom Browder
> <to...@gmail.com> wrote:
>> I have been using server side includes since I started my websites on Apache
> ...
>> Any suggestions for SSI replacement with a more asynchronous method?
>
> Let me be more specific about the data flow I'm using with the landing
> (home) page of my websites:
>
> + the first SSI line executes a CGI program that extracts the CGI and
> SSL variables and their values. Data of interest include: requesting
> IP, email address in the SSL client certificate, time of page load
>
> + the second line executes a CGI program that generates a link to a
> page that presents the current and past statistics of the visitor
> based on the stats in the db updated with the first CGI program
>
> -Tom
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
Re: [users@httpd] Re: Alternatives to SSI (server side includes)?
Posted by "Scott A. Wozny" <sa...@hotmail.com>.
Sounds like a job for AJAX, but before throwing out the baby with the bath water I'd seriously consider turning up logging with timestamps on your existing CGI and seeing if SSI is a loser in its entirety or if it's one specific part of the process that is tanking your page load times that you can easily fix (you don't say much about that data store, but if you've crossed a tipping in regards to size or number of records, you may need to reconsider your data storage strategy).
Also, if you've got really old / dumb clients AJAX might not be a great answer whereas SSI is always entirely under your control.
Best of luck,
Scott
________________________________
From: Tom Browder <to...@gmail.com>
Sent: October 3, 2020 2:08 PM
To: users@httpd.apache.org <us...@httpd.apache.org>
Subject: [users@httpd] Re: Alternatives to SSI (server side includes)?
On Sat, Oct 3, 2020 at 12:18 Tom Browder <to...@gmail.com> wrote:
> I have been using server side includes since I started my websites on Apache
...
> Any suggestions for SSI replacement with a more asynchronous method?
Let me be more specific about the data flow I'm using with the landing
(home) page of my websites:
+ the first SSI line executes a CGI program that extracts the CGI and
SSL variables and their values. Data of interest include: requesting
IP, email address in the SSL client certificate, time of page load
+ the second line executes a CGI program that generates a link to a
page that presents the current and past statistics of the visitor
based on the stats in the db updated with the first CGI program
-Tom
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
RE: [users@httpd] Re: Alternatives to SSI (server side includes)?
[EXT]
Posted by James Smith <js...@sanger.ac.uk>.
There are a number of things you can do:
1) Why are you doing what you are doing - i.e. why are you updating what is displayed to the user
2) Others have suggested using AJAX, another thing you can look at here is what and how you are retrieving the data.
You can look at using a cleanup handler (sorry mod_perl user here!) which does all the work for you and writes/updates the database, and then your ajax retrieves the data...
3) You can possibly speed up code by using a write-through cache to avoid a lot of read/writes to your database
4) From what you are saying it isn't clear what you are doing - perhaps an example URL or two would be useful for us to offer some support...
James
-----Original Message-----
From: Tom Browder <to...@gmail.com>
Sent: 03 October 2020 19:08
To: users@httpd.apache.org
Subject: [users@httpd] Re: Alternatives to SSI (server side includes)? [EXT]
On Sat, Oct 3, 2020 at 12:18 Tom Browder <to...@gmail.com> wrote:
> I have been using server side includes since I started my websites on
> Apache
...
> Any suggestions for SSI replacement with a more asynchronous method?
Let me be more specific about the data flow I'm using with the landing
(home) page of my websites:
+ the first SSI line executes a CGI program that extracts the CGI and
SSL variables and their values. Data of interest include: requesting IP, email address in the SSL client certificate, time of page load
+ the second line executes a CGI program that generates a link to a
page that presents the current and past statistics of the visitor based on the stats in the db updated with the first CGI program
-Tom
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
--
The Wellcome Sanger Institute is operated by Genome Research
Limited, a charity registered in England with number 1021457 and a
company registered in England with number 2742969, whose registered
office is 215 Euston Road, London, NW1 2BE.