How to install hyphenation patterns in FOP?

[Jerzy Jalocha N]

Hi, everyone!
Since a few days, I am trying to install the OFFO hyphenation patterns
in my FOP install, but I've failed miserably, so far. I guess, I am
missing just a tiny little detail, because It seems like I've really
tried everything. (Please see details below.)

So my question is: In order to use hyphenation with the FOP processor,
what files do I have to install, and how do I tell FOP where they are
installed?

Any help is really appreciated.
Thanks!
- Jerzy


DETAILS:
==========

My system:
* Operating system: Xubuntu 9.06 Jaunty Jackalope.
* Out-of-the-box kernel: linux-image-2.6.28-16-generic (ix86)
* FOP from the Xubuntu repository: 1:0.95.dfsg-2ubuntu1
* Both Sun Java and GCJ seem to be installed, so I am not sure which
one is running. Here is the output of 'java -version':
java version "1.6.0_16"
Java(TM) SE Runtime Environment (build 1.6.0_16-b01)
Java HotSpot(TM) Client VM (build 14.2-b01, mixed mode, sharing)

In the following sample usage scenario:
/home/work/source.fo
/home/work/hyphenation/fop-hyph.jar

* The 'source.fo' file contains hyphenate="true" and language="en"
and language="es" attributes where necessary.
* The fop-hyph.jar file was downloaded from the
'offo-hyphenation-fop-stable_v1.2.zip' archive from Sourceforge.

The following instructions do not work. They produce "Couldn't find
hyphenation pattern en" and "es" messages.

$ export FOP_HYPHENATION_PATH=/home/work/hyphenation/fop-hyph.jar
$ fop source.fo result.pdf

$ export CLASSPATH=$CLASSPATH:/home/work/hyphenation/fop-hyph.jar
$ fop source.fo result.pdf

I also tried other paths, such as 'hyphenation',
'hyphenation/fop-hyph.jar', './hyphenation/fop-hyph.jar', etc.

I also tried putting the FOP_HYPHENATION_PATH variable in a
/home/work/.foprc file without success.

The documentation also tells me to install the 'fop-hyph.jar' in a
"lib directory of the your FOP installation" or in a "build directory
of the your FOP installation". But I can't find such directories. (I
don't know enough about Java to understand what/where they are.)

Since the documentation also mentions a 'hyphenation-base'
configuration parameter, I tried putting absolute and relative paths
to 'fop-hyph.jar' and 'hyphenation' in a
'/home/work/hyphenation/fop.xconf' file. But I only got huge error
messages.

I also downloaded the 'offo-hyphenation_v1.2.zip' archive from
Sourceforge, and tried to point the 'hyphenation-base' parameter to
the 'en.xml' and 'es.xml' files, but FOP complained about missing
'*.hyp' files (whatever these might be).


I have honestly tried to read and understand the available documentation:
* http://xmlgraphics.apache.org/fop/0.95/hyphenation.html
* http://xmlgraphics.apache.org/fop/faq.html#hyphenation-fails
* http://xmlgraphics.apache.org/fop/0.95/configuration.html
* http://offo.sourceforge.net/hyphenation/installation.html
I've also read through all relevant mailing list archives, without success.


I've also asked for help through the official Ubuntu channels without success:
https://answers.launchpad.net/ubuntu/+source/fop/+question/88559

[Peter Hancock]

Hi,
You should just need fop-hyph.jar which you can get from
http://offo.sourceforge.net/hyphenation/fop-stable/installation.html

The installation just involves putting this in the lib directory and the fop
command will place it in the classpath. Also, make sure that the
build.properties (or build-local.properties) does not declare the
user.hyph.dir property - you may have uncommented this declaration whilst
trying to get things working.

HTH,
Peter

Post by Jerzy Jalocha N
Hi, everyone!
Since a few days, I am trying to install the OFFO hyphenation patterns
in my FOP install, but I've failed miserably, so far. I guess, I am
missing just a tiny little detail, because It seems like I've really
tried everything. (Please see details below.)
So my question is: In order to use hyphenation with the FOP processor,
what files do I have to install, and how do I tell FOP where they are
installed?
Any help is really appreciated.
Thanks!
- Jerzy
==========
* Operating system: Xubuntu 9.06 Jaunty Jackalope.
* Out-of-the-box kernel: linux-image-2.6.28-16-generic (ix86)
* FOP from the Xubuntu repository: 1:0.95.dfsg-2ubuntu1
* Both Sun Java and GCJ seem to be installed, so I am not sure which
java version "1.6.0_16"
Java(TM) SE Runtime Environment (build 1.6.0_16-b01)
Java HotSpot(TM) Client VM (build 14.2-b01, mixed mode, sharing)
/home/work/source.fo
/home/work/hyphenation/fop-hyph.jar
* The 'source.fo' file contains hyphenate="true" and language="en"
and language="es" attributes where necessary.
* The fop-hyph.jar file was downloaded from the
'offo-hyphenation-fop-stable_v1.2.zip' archive from Sourceforge.
The following instructions do not work. They produce "Couldn't find
hyphenation pattern en" and "es" messages.
$ export FOP_HYPHENATION_PATH=/home/work/hyphenation/fop-hyph.jar
$ fop source.fo result.pdf
$ export CLASSPATH=$CLASSPATH:/home/work/hyphenation/fop-hyph.jar
$ fop source.fo result.pdf
I also tried other paths, such as 'hyphenation',
'hyphenation/fop-hyph.jar', './hyphenation/fop-hyph.jar', etc.
I also tried putting the FOP_HYPHENATION_PATH variable in a
/home/work/.foprc file without success.
The documentation also tells me to install the 'fop-hyph.jar' in a
"lib directory of the your FOP installation" or in a "build directory
of the your FOP installation". But I can't find such directories. (I
don't know enough about Java to understand what/where they are.)
Since the documentation also mentions a 'hyphenation-base'
configuration parameter, I tried putting absolute and relative paths
to 'fop-hyph.jar' and 'hyphenation' in a
'/home/work/hyphenation/fop.xconf' file. But I only got huge error
messages.
I also downloaded the 'offo-hyphenation_v1.2.zip' archive from
Sourceforge, and tried to point the 'hyphenation-base' parameter to
the 'en.xml' and 'es.xml' files, but FOP complained about missing
'*.hyp' files (whatever these might be).
* http://xmlgraphics.apache.org/fop/0.95/hyphenation.html
* http://xmlgraphics.apache.org/fop/faq.html#hyphenation-fails
* http://xmlgraphics.apache.org/fop/0.95/configuration.html
* http://offo.sourceforge.net/hyphenation/installation.html
I've also read through all relevant mailing list archives, without success.
https://answers.launchpad.net/ubuntu/+source/fop/+question/88559
---------------------------------------------------------------------

[Jerzy Jalocha N]

Dear Peter, thank you very much for your reply!

You should just need  fop-hyph.jar  which you can get from
http://offo.sourceforge.net/hyphenation/fop-stable/installation.html

OK, this is the file I have mainly been working with, so I should be
on the right track here.

The installation just involves putting this in the lib directory

Sadly (as I explained in my first mail) I can't find this directory in
my PC. I searched for any java/lib, or lib/java or lib/fop or fop/lib
directories without success.

and the fop command will place it in the classpath.

OK, that's great, once I know which is the lib directory.

Also, make sure that the build.properties (or build-local.properties) does not declare the
user.hyph.dir property - you may have uncommented this declaration whilst
trying to get things working.

I guess, these build.properties would apply for people who compile
their stuff, but as explained, I installed directly from the Ubuntu
repository.

So it seems, the only thing I am missing here is the location of the
"lib" directory?

Just in case, I post the content of the fop script below. Maybe this helps.

Thanks!
-Jerzy


contents of /usr/bin/fop
==========
#! /bin/sh

# Shell script wrapper around the fop program,
# Copyright 2008 by Vincent Fourmond <***@debian.org>
#
# Licensed under the same terms as fop itself, that is under
# the conditions of the Apache 2 licencee.

# Include the wrappers utility script
. /usr/lib/java-wrappers/java-wrappers.sh


# We prefer to use openjdk or Sun's java if available
find_java_runtime openjdk sun || find_java_runtime

find_jars commons-io avalon-framework serializer xalan2 xml-apis
find_jars batik-all commons-logging servlet-api xercesImpl xmlgraphics-commons
find_jars xml-apis-ext
find_jars fop


# comment this line if you want fop to run without headless property
HEADLESS=-Djava.awt.headless=true

run_java $HEADLESS org.apache.fop.cli.Main "$@"

[Simon Pepping]

Post by Jerzy Jalocha N
Dear Peter, thank you very much for your reply!

You should just need  fop-hyph.jar  which you can get from
http://offo.sourceforge.net/hyphenation/fop-stable/installation.html

OK, this is the file I have mainly been working with, so I should be
on the right track here.

You are right, on Debian (and Ubuntu) systems these instructions do
not work.

On my debian system the fop script is a little different from yours:

diff of /usr/bin/fop with your fop script
==========
find_jars xml-apis-ext
+
+# We load the hyphenation jar at the request of the user.
+if [ "$FOP_HYPHENATION_PATH" ]; then
+ find_jars $FOP_HYPHENATION_PATH
+fi
find_jars fop

4 lines were added. This addition allows you to put fop-hyph.jar
wherever you like, e.g. in /usr/local/lib/java/fop-hyph.jar, or in
~/fop-hyph.jar, and set

export FOP_HYPHENATION_PATH=/usr/local/lib/java/fop-hyph.jar

or similar (the value should point to your fop-hyph.jar) in a startup
script, e.g. in .bash_profile. (I do not know how to do this when you
launch fop from a menu item; if you are lucky, .bash_profile works in
that situation as well.) Alternatively, you may start fop as:

FOP_HYPHENATION_PATH=/usr/local/lib/java/fop-hyph.jar fop args...

You may wish to raise a bug report to Ubuntu, because apparently they
failed to copy these additional lines from Debian.

I am not sure how I should add this to the OFFO installation
instructions, because it is so Debian and Ubuntu specific.

Regards, Simon

[Jerzy Jalocha N]

Simon, even if your instructions didn't work, you helped me to get it
finally working!

Post by Simon Pepping
You are right, on Debian (and Ubuntu) systems these instructions do
not work.
diff of /usr/bin/fop with your fop script
==========
 find_jars xml-apis-ext
+
+# We load the hyphenation jar at the request of the user.
+if [ "$FOP_HYPHENATION_PATH" ]; then
+    find_jars $FOP_HYPHENATION_PATH
+fi
 find_jars fop
4 lines were added. This addition allows you to put fop-hyph.jar
wherever you like, e.g. in /usr/local/lib/java/fop-hyph.jar, or in
~/fop-hyph.jar, and set
export FOP_HYPHENATION_PATH=/usr/local/lib/java/fop-hyph.jar

Strangely, this did not work. I added these lines to my /usr/bin/fop
script, but even after successfully exporting FOP_HYPHENATION_PATH to
the correct location, FOP still couldn't find the patterns.

But your patch led me to inspect the
/usr/lib/java-wrappers/java-wrappers.sh script, where I discovered
that I could use JAVA_CLASSPATH instead. Thus, the following lines
finally DO WORK as expected:

$ export JAVA_CLASSPATH=$JAVA_CLASSPATH:/home/work/hyphenation/fop-hyph.jar
$ fop source.fo result.pdf

I don't know if this is "elegant", safe, or recommended practice. But
it does the job perfectly for me, as I use FOP exclusively from
command-line and bash scripts, and don't need to launch from any
menus.

Since I am currently in contact with the Ubuntu packager, I will tell
him the results.

Thank you very much, Simon!

Regards, Jerzy

[Vincent Hennebert]

Hi Jerzy,

Post by Jerzy Jalocha N
Simon, even if your instructions didn't work, you helped me to get it
finally working!

Post by Simon Pepping
You are right, on Debian (and Ubuntu) systems these instructions do
not work.
diff of /usr/bin/fop with your fop script
==========
find_jars xml-apis-ext
+
+# We load the hyphenation jar at the request of the user.
+if [ "$FOP_HYPHENATION_PATH" ]; then
+ find_jars $FOP_HYPHENATION_PATH
+fi
find_jars fop
4 lines were added. This addition allows you to put fop-hyph.jar
wherever you like, e.g. in /usr/local/lib/java/fop-hyph.jar, or in
~/fop-hyph.jar, and set
export FOP_HYPHENATION_PATH=/usr/local/lib/java/fop-hyph.jar

Strangely, this did not work. I added these lines to my /usr/bin/fop
script, but even after successfully exporting FOP_HYPHENATION_PATH to
the correct location, FOP still couldn't find the patterns.
But your patch led me to inspect the
/usr/lib/java-wrappers/java-wrappers.sh script, where I discovered
that I could use JAVA_CLASSPATH instead. Thus, the following lines
$ export JAVA_CLASSPATH=$JAVA_CLASSPATH:/home/work/hyphenation/fop-hyph.jar
$ fop source.fo result.pdf
I don't know if this is "elegant", safe, or recommended practice. But

Neither of those :-)

The Debian/Ubuntu developers don’t use the shell script provided with
FOP. They wrote their own script (which, BTW, is quite neat) that is
adapted to the way Java is handled on those platforms. For example,
instead of shipping the dependencies with the FOP package, they made
separate packages for each of them (Apache Commons IO, Batik, XML
Graphics Commons, etc.). That way, if other packages share common
dependencies, those will be installed only once and not duplicated all
over the system.

Now, there appears to be a missing package relating to the hyphenation
patterns. What you should ask the Debian/Ubuntu developers is to create
one and make the FOP package depend on it (more precisely, ‘recommend’
it).

As a user you shouldn’t have to modify any file installed by the package
system (other than configuration files).

Post by Jerzy Jalocha N
it does the job perfectly for me, as I use FOP exclusively from
command-line and bash scripts, and don't need to launch from any
menus.
Since I am currently in contact with the Ubuntu packager, I will tell
him the results.
Thank you very much, Simon!
Regards, Jerzy

HTH,
Vincent

[Jerzy Jalocha N]

Thank you, Vincent, for taking the time to clarify this not-really-a-FOP-issue!

Post by Vincent Hennebert
The Debian/Ubuntu developers don’t use the shell script provided with
FOP. They wrote their own script (which, BTW, is quite neat) that is
adapted to the way Java is handled on those platforms. For example,
instead of shipping the dependencies with the FOP package, they made
separate packages for each of them (Apache Commons IO, Batik, XML
Graphics Commons, etc.). That way, if other packages share common
dependencies, those will be installed only once and not duplicated all
over the system.
Now, there appears to be a missing package relating to the hyphenation
patterns.

Sadly, it seems like they haven't been able to package the OFFO
hyphenation patterns because of licensing issues. Even though the
description in http://offo.sourceforge.net/hyphenation/licenses.html
states that "all files hosted by the OFFO project come with an
OSI-approved Open Source license...", closer inspection reveals that
many languages have received customizations with additional
restrictions. For example en_US.xml, which 'can be used freely for
non-commercial purposes'.

(In my opinion, these modifications should not have been accepted in OFFO.)

Post by Vincent Hennebert
What you should ask the Debian/Ubuntu developers is to create
one and make the FOP package depend on it (more precisely, ‘recommend’
it).

I will forward your comments to the FOP package manager.

Post by Vincent Hennebert
As a user you shouldn’t have to modify any file installed by the package
system (other than configuration files).

Sure, you're right! So I'll stick to my
unelegant/unsafe/not-recommended use of JAVA_CLASSPATH meanwhile. ;)

Cheers
-Jerzy

[Simon Pepping]

Post by Jerzy Jalocha N
Sadly, it seems like they haven't been able to package the OFFO
hyphenation patterns because of licensing issues. Even though the
description in http://offo.sourceforge.net/hyphenation/licenses.html
states that "all files hosted by the OFFO project come with an
OSI-approved Open Source license...", closer inspection reveals that
many languages have received customizations with additional
restrictions. For example en_US.xml, which 'can be used freely for
non-commercial purposes'.

Do they have a report on that? Where can I find that judgement? I will
try to persuade them include a hyphenation jar.

Post by Jerzy Jalocha N
(In my opinion, these modifications should not have been accepted in OFFO.)

People can use them, so I make them available. One can always remove
certain patterns from the jar file.

Simon

[Jerzy Jalocha N]

Post by Simon Pepping

Post by Jerzy Jalocha N
Sadly, it seems like they haven't been able to package the OFFO
hyphenation patterns because of licensing issues. Even though the
description in http://offo.sourceforge.net/hyphenation/licenses.html
states that "all files hosted by the OFFO project come with an
OSI-approved Open Source license...", closer inspection reveals that
many languages have received customizations with additional
restrictions. For example en_US.xml, which 'can be used freely for
non-commercial purposes'.

Do they have a report on that? Where can I find that judgement?

Here's a comment in this Ubuntu bug report
(https://bugs.launchpad.net/ubuntu/+source/fop/+bug/310882/comments/12).

Post by Simon Pepping
I will
try to persuade them include a hyphenation jar.

Great! Maybe some help will be welcome.

Thank you, Simon, and regards,
-Jerzy