Re: [xep-support] Splitting of large document

The biggest memory consumer seems to be the process that compresses the
PDF document. I find that our document requires more than 3GB to
process with this option enabled and less than 1.5GB without it.

Try the turning this option off in your xep.xml file as shown below:

    <!-- Backend options -->
    <generator-options format="PDF">
      <!-- <option name="COMPRESS" value="false"/> -->
      <!-- <option name="PDF_VERSION" value="1.3"/> -->
    </generator-options>
     
By the way, I have a problem similar to yours. Our 2200 page book is
only one part of a larger catalog. Unfortunately, the other sections
contain references to products in the main section. I had posted some
messages here earlier to see if there is a way to write page numbers and
other data into the XEP log. I thought I would extract the data with an
editor and put it in a database, using it later to look up page
references for the other sections. I don't think this is possible, but
someone suggested looking at the XEP intermediate format. I have not
done this yet. If you look at it and can distill the information I
would like to know how to do it. If I try it first, I will post here.

BJB

Jost Klopfstein wrote:

>Mike, Brian,
>
>Thanks for your hints.
>
>The document is in the 1000+ pages class with plenty of large SVG drawings
>(>500 Kbytes).
>
>I will check with my customer if I can drop the references between chapters.
>If so, then I can use the split technology and build the TOC and Indexes
>from the separate sections in XEP intermediate format.
>Otherwise they have to buy a bigger server...
>
>Does anyone know if there is an option to prevent XEP's in memory
>processing?
>
>Cheers, Jost
>
>----- Original Message -----
>From: "Mike Trotman" <mike.trotman@datalucid.com>
>To: <xep-support@renderx.com>
>Sent: Friday, June 03, 2005 3:05 AM
>Subject: Re: [xep-support] Splitting of large document
>
>
>
>
>>I have successfully processed 100mB+ documents of 1000+ pages - mainly
>>consisting of heavily formatted tables with 15 x 20 cells per page,
>>multiple pages per table, lots of data per cell, footnotes etc.
>>This included bookmarks and a simple Table Of Contents with internal
>>links to individual tables.
>>
>>By placing each table / document chunk within a separate
>><fo:page-sequence> I was able to keep the memory requirements very low
>>(not much more than the default).
>>I'm now also using XSLT pre-processing where I produce each
>><fo:page-sequence> in a separate XSL-FO file and generate a master
>>processing document which sets up regions and page masters
>>and contains a list of the separate <fo:page-sequence> files to include.
>>I then process this master list with a simple XSLT to produce the final
>>FO for output to PDF.
>>
>>I haven't used indexes (the TOC references etc. are constructed by the
>>XSLT) - so don't know what sort of overhead this produces.
>>
>>
>>Mike
>>
>>Brian J. Butler wrote:
>>
>>
>>
>>>I have also been working on a very large document (88MB FO file, 2200
>>>pages of technical text and drawings). I can offer the following three
>>>suggestions:
>>>
>>>1. Make sure your Java -Xmx size is as large as possible. With
>>>Windows this will be approximately -Xmx1600Mb.
>>>2. Use the XEP flag to turn off PDF compression (in xep.xml or command
>>>line). This will result in a very large PDF, but you can compress it
>>>after rendering by opening it in Adobe Acrobat and then saving.
>>>3. Switch to a 64-bit Solaris platform (Opteron processors). We
>>>benchmarked one of these machines and found that we can -Xmx almost
>>>unlimited memory. The speed is also very fast.
>>>
>>>BJB
>>>
>>>Jost Klopfstein wrote:
>>>
>>>
>>>
>>>>Hi,
>>>>
>>>>I ran into memory problems while rendering a large book with TOC,
>>>>indexes and references between sections.
>>>>I first thought I could just render section by section into XEP
>>>>intermediate format and then assemble the pieces with some custom
>>>>code into a large PDF using the PDF output generator.
>>>>However I will loose the TOC, indexes and the references between
>>>>sections.
>>>>
>>>>Any ideas?
>>>>
>>>>Thanks,
>>>>Jost
>>>>
>>>>
>>>>
>>>------------------------------------------------------------------------
>>>
>>>
>>>>Jost Klopfstein
>>>>*Axos Technologies Inc.*
>>>>OnDemand & Transactional Document Solutions, powered by XML
>>>>IT Consulting
>>>>
>>>>*604 628-2248 Phone*
>>>>604-324-2380 Fax
>>>>jost (at) axostech.com
>>>>http://www.axostech.com
>>>>
>>>>
>>>--
>>>Brian J. Butler
>>>BJB Software, Inc.
>>>76 Bayberry Lane
>>>Holliston, MA 01746
>>>
>>>E-mail: bjbutler@bjbsoftware.com
>>>Web: http://www.bjbsoftware.com
>>>Phone: 508-429-1441
>>>Fax: 419-710-1867
>>>
>>>
>>>
>>>
>>>
>>
>>
>>--
>>No virus found in this outgoing message.
>>Checked by AVG Anti-Virus.
>>Version: 7.0.322 / Virus Database: 267.5.2 - Release Date: 03/06/2005
>>
>>
>>Message Scanned by ClamAV on datalucid.com
>>-------------------
>>(*) To unsubscribe, send a message with words 'unsubscribe xep-support'
>>in the body of the message to majordomo@renderx.com from the address
>>you are subscribed from.
>>(*) By using the Service, you expressly agree to these Terms of Service
>>
>>
>http://www.renderx.com/terms-of-service.html
>
>
>>
>>
>
>-------------------
>(*) To unsubscribe, send a message with words 'unsubscribe xep-support'
>in the body of the message to majordomo@renderx.com from the address
>you are subscribed from.
>(*) By using the Service, you expressly agree to these Terms of Service http://www.renderx.com/terms-of-service.html
>
>
>
>
>

-- 
Brian J. Butler
BJB Software, Inc.
76 Bayberry Lane
Holliston, MA 01746
E-mail: bjbutler@bjbsoftware.com
Web:    http://www.bjbsoftware.com
Phone:  508-429-1441
Fax:    419-710-1867
-------------------
(*) To unsubscribe, send a message with words 'unsubscribe xep-support'
in the body of the message to majordomo@renderx.com from the address
you are subscribed from.
(*) By using the Service, you expressly agree to these Terms of Service http://www.renderx.com/terms-of-service.html