Discussion:
Re[2]: FOP being very slow.
Jignesh-NX01880 Kapadia
2004-01-12 15:39:22 UTC
Permalink
Hi Chris & John ,

Thanks a lot for your responses. I' m answering your questions and the info
you have asked as below.

For Chris

This application was developed by third party for us and delivered to us. Now
since it is very slow we need to modify it to make it less memory intensive. The
approach they have used is as follows.

1) The data retrieved from database is converted in to XML.

2) That XML file with the help of -it is converted in PDF.

3) The FOP version used is 0.20.4rc


For John,

we do have load balancing of application servers. Both the servers have single
CPU and JVM size on each can be extended to 1GB at max. We have XSLT
transformations also as I mentioned above in Chris's response.

Do you guys think 0.20.5 will have a better performance time then 0.20.4rc.

I am attaching a sample code from one of the main method which generates PDF
file from our PDF Servlet. and please do suggest any alternative if you think it
can be done in better way to improve processing speed.





and then it is written to output as follows:



byte[] content = generate(contextRoot, contextRootURL, generateParams,
templateType, templateName,
targetLanguage, xmlPriceBook, cover, pdfCharts, false);

// session.setAttribute("LAST_PDF", content);
File outFile = new File(baseDirectory + "cache/" +
userProfile.getUserId() + ".pdf");
FileOutputStream fos = new FileOutputStream(outFile);
fos.write(content);
fos.close();

response.sendRedirect(baseHTTP_URL + "cache/" +
userProfile.getUserId() + ".pdf");

Thanks and your help will be really appreciated,

Jignesh


____________________Reply Separator____________________
Subject: Re: FOP being very slow.
Author: ***@hotmail.com
Date: 1/12/2004 9:18 AM
Hi ,
We are having here. We are using FOP here for our application it
generates
a 81 page PDF in the output. It takes a very long time to generate it almost
more than 1 minute.
What sort of system are you running? how many CPUs, what o/s, (you
mentioned memory below), and what version of FOP?

The figures you quote sound reasonsable for a basic single CPU system. I
dont think you can do a lot to speed up processing time for a single
document. Actually I found FOP to be one of the faster XSL-FO formatters
around (mainly due to the fact that keeps and other difficult features
are missing)
The user acceptance criteria is 15-20 seconds for such big
PDF. if say more than 20 user try to generate the PDF file then we reach peak
with our JVM and after that no user can do anything. We are using following
FOP
driver class.
org.w3c.dom.svg.SVGDocument.
I dont quite follow this bit. Are you saying you are creating SVG and
XSL-FO in a DOM before presenting it to FOP for processing. This seems
somewhat inefficient.
The websphere instance on which the application is running has 512MB of JVM
size. We need that application can sustain a load of 60 user at a time to
generate PDF document. The server has RAM of 3GB. The application is built
with
struts. Is there any way that FOP is can be made less memory intensive? so
that
it will not use much JVM but use CPU rather?
There have been some changes made to improve the memory consumption of
tables, but these changes have not been included in any release of FOP.
You will have to download the source from CVS maintenance branch and
compile it yourself.

Chris



---------------------------------------------------------------------
To unsubscribe, e-mail: fop-user-***@xml.apache.org
For additional commands, e-mail: fop-user-***@xml.apache.org
John Austin
2004-01-12 15:55:30 UTC
Permalink
Post by Jignesh-NX01880 Kapadia
Hi Chris & John ,
Thanks a lot for your responses. I' m answering your questions and the info
you have asked as below.
For Chris
This application was developed by third party for us and delivered to us. Now
since it is very slow we need to modify it to make it less memory intensive. The
approach they have used is as follows.
1) The data retrieved from database is converted in to XML.
2) That XML file with the help of -it is converted in PDF.
3) The FOP version used is 0.20.4rc
Of more interest, which JVM and what version are you using ?

Does it employ 'java -server' version of the JVM ?
Post by Jignesh-NX01880 Kapadia
For John,
we do have load balancing of application servers. Both the servers have single
CPU and JVM size on each can be extended to 1GB at max. We have XSLT
transformations also as I mentioned above in Chris's response.
Do you guys think 0.20.5 will have a better performance time then 0.20.4rc.
It's worth measuring in your environment.
Post by Jignesh-NX01880 Kapadia
I am attaching a sample code from one of the main method which generates PDF
file from our PDF Servlet. and please do suggest any alternative if you think it
can be done in better way to improve processing speed.
Following 'generate' is part of the package written for you ?

I don't recognize it off the top of my head.
Post by Jignesh-NX01880 Kapadia
byte[] content = generate(contextRoot, contextRootURL, generateParams,
templateType, templateName,
targetLanguage, xmlPriceBook, cover, pdfCharts, false);
// session.setAttribute("LAST_PDF", content);
File outFile = new File(baseDirectory + "cache/" +
userProfile.getUserId() + ".pdf");
FileOutputStream fos = new FileOutputStream(outFile);
fos.write(content);
fos.close();
response.sendRedirect(baseHTTP_URL + "cache/" +
userProfile.getUserId() + ".pdf");
So you build the file, write to disk and then redirect the user to
that file. Useful only if the user access the result more than once
AND there isn't any caching on your server or the browser.
Post by Jignesh-NX01880 Kapadia
Thanks and your help will be really appreciated,
Jignesh
I would look again at the FOP Servlet examples and also look at the XSLT
transformations to make sure they are not poor performers.
Post by Jignesh-NX01880 Kapadia
____________________Reply Separator____________________
Subject: Re: FOP being very slow.
Date: 1/12/2004 9:18 AM
Hi ,
We are having here. We are using FOP here for our application it
generates
a 81 page PDF in the output. It takes a very long time to generate it almost
more than 1 minute.
What sort of system are you running? how many CPUs, what o/s, (you
mentioned memory below), and what version of FOP?
The figures you quote sound reasonsable for a basic single CPU system. I
dont think you can do a lot to speed up processing time for a single
document. Actually I found FOP to be one of the faster XSL-FO formatters
around (mainly due to the fact that keeps and other difficult features
are missing)
The user acceptance criteria is 15-20 seconds for such big
PDF. if say more than 20 user try to generate the PDF file then we reach peak
with our JVM and after that no user can do anything. We are using following
FOP
driver class.
org.w3c.dom.svg.SVGDocument.
I dont quite follow this bit. Are you saying you are creating SVG and
XSL-FO in a DOM before presenting it to FOP for processing. This seems
somewhat inefficient.
The websphere instance on which the application is running has 512MB of JVM
size. We need that application can sustain a load of 60 user at a time to
generate PDF document. The server has RAM of 3GB. The application is built
with
struts. Is there any way that FOP is can be made less memory intensive? so
that
it will not use much JVM but use CPU rather?
There have been some changes made to improve the memory consumption of
tables, but these changes have not been included in any release of FOP.
You will have to download the source from CVS maintenance branch and
compile it yourself.
Chris
---------------------------------------------------------------------
______________________________________________________________________
---------------------------------------------------------------------
--
John Austin <***@sympatico.ca>
Chris Bowditch
2004-01-12 16:06:14 UTC
Permalink
Post by Jignesh-NX01880 Kapadia
Do you guys think 0.20.5 will have a better performance time then 0.20.4rc.
I believe there is about a 10% performance improvement between 0.20.4
and 0.20.5 due to revised property handling.

Chris
m***@iworks.fr
2004-01-12 18:00:45 UTC
Permalink
I had attached one .TXT file.in my last e-mail
The code sample you sent is OK.
A remark :
"org.apache.fop.apps.Driver" : this is the FOP driver
"Class.forName("org.w3c.dom.svg.SVGDocument");" loads the SVGDocument
class. It might be used by FOP to render SVG (vector) graphics. I don't
see why it is helpful to write this line here.

As John said it before, you must determine what takes time and memory :
XSLT transformation or PDF rendering. While rendering, it is difficult to
know what happens, but you should try to desactivate charts and see what
happens. If your charts are SVGs, you should try plainer shapes.

Mathieu.
Jignesh-NX01880 Kapadia
2004-01-12 22:22:43 UTC
Permalink
Hi John,
Well we are running this application on UNIX machines. From where did you get
hint that its running on windows.. Then we can use the " java -server"
optimizing JVM. But where exactly to use it.

Regards,

Jignesh

____________________Reply Separator____________________
Subject: Re: FOP being very slow.
Author: ***@sympatico.ca
Date: 1/12/2004 4:48 PM
I had attached one .TXT file.in my last e-mail. That was the main
generate method which generates the output to write in to a .PDF file. I am
attaching again.
Doh! Sorry I missed that.
From the listing I note you are running on Windows, so
you won't be able to use the 'java -server' optimizing JVM.

Performance hint:

The Cocoon people say that logging kills performance. Be prepared to
turn it down if not completely off.

One item that can be changed in your program:

Instead of writing the transformer output to a file using the
second arg to transformer.transform()
and feeding the written XML file back in through driver.setInputSource()

Try the following:

File xsltFile2 = new File( xsltFileName2);

Source xsltSource2 = new StreamSource(xsltFile2);
Transformer trans2 = transFact.newTransformer( xsltSource2 );

Source xmlSource2 = new StreamSource(out1);

//*** Generate PDF file ***

File pdfFile = new File( namePart + ".pdf" );

Driver driver = new Driver();

driver.setRenderer( Driver.RENDER_PDF );

try {
driver.setOutputStream( new java.io.FileOutputStream( pdfFile ) );
Result res = new SAXResult(driver.getContentHandler());
trans2.transform( xmlSource2, res );
}
catch( IOException ioe ) {
From your listing I note that you recreate the Transformer and Driver
objects for every invocation. (In a servlet) It is possibile to re-use
the XSL transformation which may or may not be useful to you. I can't
tell you how much benefit that would provide because I have to work from
measurements of execution times of your XSLT vs your FOP formatting.

For comparison, I have a program which reads an XML file generated from
a database and applies an XSLT transformation which produces an XSL-FO
document. Rather than create the XSL-FO file, my program sends the
XSLT output to the FOP driver.

This program read an XSL file representing 400 names and addresses and
generated a customized form letter in a PDF file of some 800 pages.
As I recall the program ran in less than one minute on a 800 MHz Athlon
with 512Mb SDRAM.
Subject: Re: Re[2]: FOP being very slow.
Date: 1/12/2004 10:52 AM
Post by Jignesh-NX01880 Kapadia
Hi Chris & John ,
Thanks a lot for your responses. I' m answering your questions and the
info
Post by Jignesh-NX01880 Kapadia
you have asked as below.
For Chris
This application was developed by third party for us and delivered to us.
Now
Post by Jignesh-NX01880 Kapadia
since it is very slow we need to modify it to make it less memory intensive.
The
Post by Jignesh-NX01880 Kapadia
approach they have used is as follows.
1) The data retrieved from database is converted in to XML.
2) That XML file with the help of -it is converted in PDF.
3) The FOP version used is 0.20.4rc
Of more interest, which JVM and what version are you using ?
Does it employ 'java -server' version of the JVM ?
Post by Jignesh-NX01880 Kapadia
For John,
we do have load balancing of application servers. Both the servers have
single
Post by Jignesh-NX01880 Kapadia
CPU and JVM size on each can be extended to 1GB at max. We have XSLT
transformations also as I mentioned above in Chris's response.
Throwing hardware at a slow implementation is not my favorite solution
but it is always an option. You can get an inexpensive Linux box with
dual processors, a fast disk and several Gigs of DDR memory for under
USD $3000.
Post by Jignesh-NX01880 Kapadia
Do you guys think 0.20.5 will have a better performance time then 0.20.4rc.
It's worth measuring in your environment.
Post by Jignesh-NX01880 Kapadia
I am attaching a sample code from one of the main method which generates PDF
file from our PDF Servlet. and please do suggest any alternative if you
think
it
Post by Jignesh-NX01880 Kapadia
can be done in better way to improve processing speed.
Following 'generate' is part of the package written for you ?
I don't recognize it off the top of my head.
Post by Jignesh-NX01880 Kapadia
byte[] content = generate(contextRoot, contextRootURL, generateParams,
templateType, templateName,
targetLanguage, xmlPriceBook, cover, pdfCharts, false);
// session.setAttribute("LAST_PDF", content);
File outFile = new File(baseDirectory + "cache/" +
userProfile.getUserId() + ".pdf");
FileOutputStream fos = new FileOutputStream(outFile);
fos.write(content);
fos.close();
response.sendRedirect(baseHTTP_URL + "cache/" +
userProfile.getUserId() + ".pdf");
So you build the file, write to disk and then redirect the user to
that file. Useful only if the user access the result more than once
AND there isn't any caching on your server or the browser.
Post by Jignesh-NX01880 Kapadia
Thanks and your help will be really appreciated,
Jignesh
I would look again at the FOP Servlet examples and also look at the XSLT
transformations to make sure they are not poor performers.
Post by Jignesh-NX01880 Kapadia
____________________Reply Separator____________________
Subject: Re: FOP being very slow.
Date: 1/12/2004 9:18 AM
Hi ,
We are having here. We are using FOP here for our application it
generates
a 81 page PDF in the output. It takes a very long time to generate it
almost
Post by Jignesh-NX01880 Kapadia
more than 1 minute.
What sort of system are you running? how many CPUs, what o/s, (you
mentioned memory below), and what version of FOP?
The figures you quote sound reasonsable for a basic single CPU system. I
dont think you can do a lot to speed up processing time for a single
document. Actually I found FOP to be one of the faster XSL-FO formatters
around (mainly due to the fact that keeps and other difficult features
are missing)
The user acceptance criteria is 15-20 seconds for such big
PDF. if say more than 20 user try to generate the PDF file then we reach
peak
Post by Jignesh-NX01880 Kapadia
with our JVM and after that no user can do anything. We are using
following
Post by Jignesh-NX01880 Kapadia
FOP
driver class.
org.w3c.dom.svg.SVGDocument.
I dont quite follow this bit. Are you saying you are creating SVG and
XSL-FO in a DOM before presenting it to FOP for processing. This seems
somewhat inefficient.
The websphere instance on which the application is running has 512MB of
JVM
Post by Jignesh-NX01880 Kapadia
size. We need that application can sustain a load of 60 user at a time to
generate PDF document. The server has RAM of 3GB. The application is built
with
struts. Is there any way that FOP is can be made less memory intensive? so
that
it will not use much JVM but use CPU rather?
There have been some changes made to improve the memory consumption of
tables, but these changes have not been included in any release of FOP.
You will have to download the source from CVS maintenance branch and
compile it yourself.
Chris
______________________________________________________________________
--
John Austin <***@sympatico.ca>

---------------------------------------------------------------------
To unsubscribe, e-mail: fop-user-***@xml.apache.org
For additional commands, e-mail: fop-user-***@xml.apache.org
Roland Neilands
2004-01-12 23:26:09 UTC
Permalink
Post by Jignesh-NX01880 Kapadia
Well we are running this application on UNIX machines. From where did you get
hint that its running on windows.. Then we can use the " java -server"
optimizing JVM. But where exactly to use it.
fop.sh, last line, just add " -server" after "java"

Cheers,
Roland
John Austin
2004-01-13 00:06:38 UTC
Permalink
Post by Roland Neilands
Post by Jignesh-NX01880 Kapadia
Well we are running this application on UNIX machines. From where did you get
hint that its running on windows.. Then we can use the " java -server"
optimizing JVM. But where exactly to use it.
fop.sh, last line, just add " -server" after "java"
I forgot that he is running in websphere. Anyone know which servlet
container is used by websphere ? Does it use the SUN or IBM SDK ?

[Sent him a program offline ... and explained that I saw a DOS
(I mean WINDOWS) file name in his Code.]
--
John Austin <***@sympatico.ca>
John Austin
2004-01-13 18:33:44 UTC
Permalink
On Mon, 2004-01-12 at 20:36, John Austin wrote:

In summary, I sent him my sample program and then reworked
some of that program to eliminate communication through files.

The sample program reads an external XML file generated from an
Access DataBase.

Transformation 1 generates a DOMResult which contains up to
about 400 records from the DB. They only add about 1k each to
the memory requirements.

Transformation 2 reads a new DOMSource( theDOMResult.getNode() );
and generates a SAXResult() that uses the FOP Driver to write
a PDF file.

This demonstrates enough performance improvements to cut down
his execution time significantly.

I also pointed out that LOGGING is a performance hog. This is
mentioned on the Cocoon list from time to time. In one case
I got a 20% improvement for an 80 page PDF.

When he absorbs this, (and if he comes back) I might point him
towards some material on servlets and re-using his compiled
XSLT stylesheets.
--
John Austin <***@sympatico.ca>
Clay Leeds
2004-01-13 19:02:04 UTC
Permalink
Post by John Austin
I also pointed out that LOGGING is a performance hog. This is
mentioned on the Cocoon list from time to time. In one case
I got a 20% improvement for an 80 page PDF.
Just out of curiosity, is there a way (perhaps via avalon?) of logging
every nth (10th, 100th, 1000th, etc.) rendering? That way, one might be
able to gather aggregate data, but not have as much of a performance
penalty.

Web Maestro Clay
Glen Mazza
2004-01-13 21:05:57 UTC
Permalink
Yes, logging appears to be overdone within FOP, even
if a NULL logger object is chosen, a lot of function
calls are still being made. I haven't researched
Xalan much yet, but their system may be a good guide
on the amount of logging appropriate for our system.

Glen
Post by John Austin
I also pointed out that LOGGING is a performance
hog. This is
mentioned on the Cocoon list from time to time. In
one case
I got a 20% improvement for an 80 page PDF.
__________________________________
Do you Yahoo!?
Yahoo! Hotjobs: Enter the "Signing Bonus" Sweepstakes
http://hotjobs.sweepstakes.yahoo.com/signingbonus

Jignesh-NX01880 Kapadia
2004-01-12 22:34:20 UTC
Permalink
Hi John,
Can you send me a sample program which you talked about in your last e-mail.

Thanks,
Jignesh

____________________Reply Separator____________________
Subject: Re: FOP being very slow.
Author: ***@sympatico.ca
Date: 1/12/2004 4:48 PM
I had attached one .TXT file.in my last e-mail. That was the main
generate method which generates the output to write in to a .PDF file. I am
attaching again.
Doh! Sorry I missed that.
From the listing I note you are running on Windows, so
you won't be able to use the 'java -server' optimizing JVM.

Performance hint:

The Cocoon people say that logging kills performance. Be prepared to
turn it down if not completely off.

One item that can be changed in your program:

Instead of writing the transformer output to a file using the
second arg to transformer.transform()
and feeding the written XML file back in through driver.setInputSource()

Try the following:

File xsltFile2 = new File( xsltFileName2);

Source xsltSource2 = new StreamSource(xsltFile2);
Transformer trans2 = transFact.newTransformer( xsltSource2 );

Source xmlSource2 = new StreamSource(out1);

//*** Generate PDF file ***

File pdfFile = new File( namePart + ".pdf" );

Driver driver = new Driver();

driver.setRenderer( Driver.RENDER_PDF );

try {
driver.setOutputStream( new java.io.FileOutputStream( pdfFile ) );
Result res = new SAXResult(driver.getContentHandler());
trans2.transform( xmlSource2, res );
}
catch( IOException ioe ) {
From your listing I note that you recreate the Transformer and Driver
objects for every invocation. (In a servlet) It is possibile to re-use
the XSL transformation which may or may not be useful to you. I can't
tell you how much benefit that would provide because I have to work from
measurements of execution times of your XSLT vs your FOP formatting.

For comparison, I have a program which reads an XML file generated from
a database and applies an XSLT transformation which produces an XSL-FO
document. Rather than create the XSL-FO file, my program sends the
XSLT output to the FOP driver.

This program read an XSL file representing 400 names and addresses and
generated a customized form letter in a PDF file of some 800 pages.
As I recall the program ran in less than one minute on a 800 MHz Athlon
with 512Mb SDRAM.
Subject: Re: Re[2]: FOP being very slow.
Date: 1/12/2004 10:52 AM
Post by Jignesh-NX01880 Kapadia
Hi Chris & John ,
Thanks a lot for your responses. I' m answering your questions and the
info
Post by Jignesh-NX01880 Kapadia
you have asked as below.
For Chris
This application was developed by third party for us and delivered to us.
Now
Post by Jignesh-NX01880 Kapadia
since it is very slow we need to modify it to make it less memory intensive.
The
Post by Jignesh-NX01880 Kapadia
approach they have used is as follows.
1) The data retrieved from database is converted in to XML.
2) That XML file with the help of -it is converted in PDF.
3) The FOP version used is 0.20.4rc
Of more interest, which JVM and what version are you using ?
Does it employ 'java -server' version of the JVM ?
Post by Jignesh-NX01880 Kapadia
For John,
we do have load balancing of application servers. Both the servers have
single
Post by Jignesh-NX01880 Kapadia
CPU and JVM size on each can be extended to 1GB at max. We have XSLT
transformations also as I mentioned above in Chris's response.
Throwing hardware at a slow implementation is not my favorite solution
but it is always an option. You can get an inexpensive Linux box with
dual processors, a fast disk and several Gigs of DDR memory for under
USD $3000.
Post by Jignesh-NX01880 Kapadia
Do you guys think 0.20.5 will have a better performance time then 0.20.4rc.
It's worth measuring in your environment.
Post by Jignesh-NX01880 Kapadia
I am attaching a sample code from one of the main method which generates PDF
file from our PDF Servlet. and please do suggest any alternative if you
think
it
Post by Jignesh-NX01880 Kapadia
can be done in better way to improve processing speed.
Following 'generate' is part of the package written for you ?
I don't recognize it off the top of my head.
Post by Jignesh-NX01880 Kapadia
byte[] content = generate(contextRoot, contextRootURL, generateParams,
templateType, templateName,
targetLanguage, xmlPriceBook, cover, pdfCharts, false);
// session.setAttribute("LAST_PDF", content);
File outFile = new File(baseDirectory + "cache/" +
userProfile.getUserId() + ".pdf");
FileOutputStream fos = new FileOutputStream(outFile);
fos.write(content);
fos.close();
response.sendRedirect(baseHTTP_URL + "cache/" +
userProfile.getUserId() + ".pdf");
So you build the file, write to disk and then redirect the user to
that file. Useful only if the user access the result more than once
AND there isn't any caching on your server or the browser.
Post by Jignesh-NX01880 Kapadia
Thanks and your help will be really appreciated,
Jignesh
I would look again at the FOP Servlet examples and also look at the XSLT
transformations to make sure they are not poor performers.
Post by Jignesh-NX01880 Kapadia
____________________Reply Separator____________________
Subject: Re: FOP being very slow.
Date: 1/12/2004 9:18 AM
Hi ,
We are having here. We are using FOP here for our application it
generates
a 81 page PDF in the output. It takes a very long time to generate it
almost
Post by Jignesh-NX01880 Kapadia
more than 1 minute.
What sort of system are you running? how many CPUs, what o/s, (you
mentioned memory below), and what version of FOP?
The figures you quote sound reasonsable for a basic single CPU system. I
dont think you can do a lot to speed up processing time for a single
document. Actually I found FOP to be one of the faster XSL-FO formatters
around (mainly due to the fact that keeps and other difficult features
are missing)
The user acceptance criteria is 15-20 seconds for such big
PDF. if say more than 20 user try to generate the PDF file then we reach
peak
Post by Jignesh-NX01880 Kapadia
with our JVM and after that no user can do anything. We are using
following
Post by Jignesh-NX01880 Kapadia
FOP
driver class.
org.w3c.dom.svg.SVGDocument.
I dont quite follow this bit. Are you saying you are creating SVG and
XSL-FO in a DOM before presenting it to FOP for processing. This seems
somewhat inefficient.
The websphere instance on which the application is running has 512MB of
JVM
Post by Jignesh-NX01880 Kapadia
size. We need that application can sustain a load of 60 user at a time to
generate PDF document. The server has RAM of 3GB. The application is built
with
struts. Is there any way that FOP is can be made less memory intensive? so
that
it will not use much JVM but use CPU rather?
There have been some changes made to improve the memory consumption of
tables, but these changes have not been included in any release of FOP.
You will have to download the source from CVS maintenance branch and
compile it yourself.
Chris
______________________________________________________________________
--
John Austin <***@sympatico.ca>

---------------------------------------------------------------------
To unsubscribe, e-mail: fop-user-***@xml.apache.org
For additional commands, e-mail: fop-user-***@xml.apache.org
Continue reading on narkive:
Loading...