Muhimbi

Posted at: 12:29 PM on 26 September 2011 by Muhimbi

Muhimbi’s range of PDF Conversion products, including the PDF Converter for SharePoint and the PDF Converter Services, has provided some level of PDF/A support since the very beginning. We knew this was going to be important so we even built it into our web service’s object model. However, up until now (version 5.1), any request to output the file in PDF/A format was passed on directly to the underlying converter, which may, or may not, do a good job of sticking to the PDF/A standard (more on this subject below).

The converters that support this kind of output natively are:

• MS-Word
• PowerPoint
• MS-Publisher
• Viso
• TIFF

This all all changes with version 5.2 of our software, which allows any PDF file to be post processed and converted to PDF/A-1b. More about this later, let’s have a look at the history, and reasons, behind the PDF/A standard first.

Over the last 20 years the PDF document format has grown from a relatively basic file type that displays information on screen as if it were printed on paper, to an all singing and dancing 3D, Audio / Video digital printing format. Although at the moment it is easy and free to download a viewer that can display the latest and greatest content, what about the computers used 50 years from now? Who knows, perhaps the holographic AppleMicroGoogle™ quantum computers we’ll be using by then won’t even understand the concept of embedded JavaScript, or no one could be bothered to implement it in the version of Acrobat Reader that was rewritten from the ground up after the Great Computer Failure of 2038. All kidding aside, the PDF/A standard was created to go ‘back to basics’ and use PDF for what it was originally intended for and give future computers and operating systems a remote chance of displaying a file’s content similar to how it was originally intended to look.

A good summary of the main aims and rules with regards to PDF/A can be found on the pdfa.org website. 

What does PDF/A aim to achieve? PDF/A aims to produce files with static content that can therefore be visually reproduced completely precisely today and in many years time. Files that are subject to long-term archiving should work regardless of the device or operating system used. The future usability of PDF/A files must also be guaranteed in a manufacturer-independent manner – and this includes Adobe. PDF/A is a ‘complete’ format. This means that PDF files that comply with the PDF/A standard are complete in themselves and use no external references or non-PDF data. The PDF/A-1 standard is based on PDF/A specification 1.4, which means that it works within the technical scope of the functions available in Acrobat 5.

A range of rules must be observed when generating PDF/A files in order to meet the goals named above. For example, when generating PDF/A, it is important to embed all fonts and clearly specify all colors. Forms, comments, and notes are only permitted to a limited extent. Compression is allowed as a general rule, but LZW and JPEG2000 are excluded. Transparent objects and layers (Optional Content Groups) are not permitted. PDF/A uses rules for metadata that are based on XMP (Extensible Metadata Platform). Finally, a PDF/A file must identify itself as such.

Please note that you need the PDF Converter Professional add-on license in addition to a valid PDF Converter for SharePoint or PDF Converter Services License in order to use this functionality.
 

Muhimbi’s interpretation of the PDF/A-1b standard

As is generally the case with these kinds of standards, there are as many PDF/A validators as there are interpretations of the specification. The validator we use internally for testing purposes is the one that comes with Adobe Acrobat Pro X. I am sure there are stricter validators, but if Acrobat Pro says a document validates fine then I am sure there is a pretty good chance it can be opened 50 years from now.

In the screenshot below you can see the validation results of a PowerPoint file that was converted to PDF by the Muhimbi PDF Converter and subsequently post processed for output as PDF/A. As you can see it validates perfectly well.
 

PDF/A File generated by the Muhimbi PDF Converter

As you can see in the screenshot below, the same document saved by PowerPoint itself in PDF/A format does not validate successfully. 
 

PDF/A File generated by PowerPoint

  
Clearly the file post-processed by the Muhimbi PDF Converter validates better, but that doesn’t mean that the one generated by PowerPoint cannot be displayed 50 years from now. The validator checks many rules including important ones such as font embedding, but also rules that are perhaps not so important like the fact that the Modification Date stored in the PDF is exactly the same as the one stored in the XMP meta-data.

Configuring the Muhimbi PDF Converter to use PDF/A

The Muhimbi PDF Converter relies on 3rd party software to carry out the PDF/A post processing step. Fortunately this software is free to download and install for both individuals and organisations. The actual use of the software is less than trivial, but our software takes care of all the complexities.

Installation of this software is optional and only required if you intend to carry out any PDF/A post processing. The steps are as follows:

1. If not already installed, install the Muhimbi PDF Converter for SharePoint version 5.2 or newer.
2. Download the latest GPL Release from the Ghostscript website. Depending on your hardware and operating system you will need to download either the 32 or 64 bit version. Muhimbi has tested the PDF/A post processing with version 9.04.
3. Install Ghostscript in a location of your choice on every server that runs the Muhimbi Conversion Service. If you accept the default location, or the default location on a different drive, then the Muhimbi PDF Converter will automatically detect Ghostscript’s file path.

Once Ghostscript has been installed you are ready to go, providing you access our Web Services based interface from your own software and the PDFA.PostProcessing configuration value discussed below is set appropriately (details for use with SharePoint can be found further below). Just set the web service’s ConversionSettings.PDProfile property to PDF_A1B and any converter that doesn’t natively support PDF/A output will automatically send the generated PDF file to the post processor. However, depending on your exact needs you may want to update a number of settings in the Muhimbi Conversion Service’s config file.

1. Open ‘Muhimbi.DocumentConverter.Service.exe.config’ in your favourite text editor. The file is stored in the same directory where the Muhimbi Conversion Service has been installed in. You can find a shortcut to this folder in the ‘Muhimbi Document Converter’ group in your start menu.
2. Change the following settings if needed:
   • Ghostscript.Path: Leave the path empty to auto detect the path. When manually specifying the path include the executable as well, e.g. "E:\Program Files\gs\gs9.04\bin\gswin64c.exe"
   • PDFA.PostProcessing:  Not all converters are able to provide native PDF/A output. Use this setting to post process any generated PDF file. Valid values are 'All' (Post Process files generated by all converters, including the ones that are supposed to already support PDF/A), 'WhenNeeded' (Post process files for only those converters that do not support native PDF/A output) or 'None' (Do not post process files generated by any converters. This is the default option). Please note that these values will only be used if the output format is set to PDF_A1B, either in the web service call or via the global 'ConversionSettings.ForcePDFProfile' config value.
   • PDFA.RasterizeTransparentContent: Define how transparent content is dealt with during conversion to PDF/A. The default setting (False) removes all transparency. If you wish to retain transparent objects then set this value to True, which will result in pages being rasterized resulting in considerably larger and slower PDF files.
   • ConversionSettings.ForcePDFProfile: Override the web service’s ConversionSettings.PDFProfile value during conversion.  Leave empty to use the setting specified in the web service call. Accepted values are members of the Muhimbi.DocumentConverter.WebService.Data.PDFProfile enumeration or an empty string. For example: 'PDF_1_5' (Use PDF Version 1.5) or 'PDF_A1B' (Use the PDF/A standard for long term archiving).
3. Restart the Muhimbi Conversion Service from the Windows Services Management Console.

If you don’t directly interact with Muhimbi’s Web Services interface, but rather use the SharePoint Front End functionality that comes with the PDF Converter for SharePoint, e.g. workflow activities or manual PDF Conversion, then you MUST set the ForcePDFProfile configuration value to PDF_A1B. Please note that that this is a global switch that forces all functionality provided by our product to output in PDF/A format. This may have side effects when applying PDF Security. In the future we will provide more granular support for those customers who rely on the SharePoint facilities that come with our products.

Sample code to convert PDF Documents to PDF/A using our web services interface can be found here. If you don’t wish to use Ghostscript, or if your organisation already uses a different PDF to PDF/A converter that you wish to integrate with, then please contact us.

Issues and limitations

At Muhimbi we like to under promise and over deliver, so interpret the following in that light. We prefer to be upfront about any issues you may encounter to prevent unexpected surprises at a later date.

At this moment in time we are aware of the following issues:

1. 64 bit only: At this moment the 32 bit version of Ghostscript (9.04) contains a bug that may interfere with PDF/A output. As a result you will need to install the 64 bit version. In other words you can only run the entire solution successfully on a 64 bit machine running a 64 bit Operating System. It is expected that this bug will be fixed in the ‘post 9.04 version’ expected at the beginning of 2012.
2. Merged Documents: When documents are merged using the Muhimbi PDF Converter, and subsequently post processed for PDF/A output, then Acrobat’s validator shows the ‘CIDSystemInfo and CMap dict not compatible' validation error. It doesn’t happen for all document types and we are confident this message does not have any significant side effects.

 
There are other side effects inherent to the PDF/A standard that we can’t do anything about. For example, as transparency is not supported, any documents that use (semi) transparent objects may not look the same as the source document. Similarly, because fonts MUST be embedded in PDF/A-1b compliant documents, the resulting PDF file may be larger than expected, although in many cases you will find them to be smaller than the source file.

We love working with our customers and we’ll do our very best to solve any problems you may experience, so please drop us a line if you have any questions or require assistance.

Interestingly the various committees have already started work on PDF/A2, which aims to put everything back in that was stripped out of PDF/A-1b (3D Graphics, Audio, Video etc). To be continued….

.

Labels: InfoPath, pdf, PDF Converter, PDF Converter Professional, PDF Converter Services, PDF/A