Revert "Decode JPEGs at reduced size via libjpeg scale_denom"

This reverts commit 009b91d9d95d998fa7e8e09c55f716cb42d99546.

Reason for revert: Signifincantly slows down rendering in some cases.

Failure Link: https://ci.chromium.org/ui/p/chromium/builders/ci/win11-arm64-dbg-tests/12279/overview

Original change's description:
> Decode JPEGs at reduced size via libjpeg scale_denom
>
> JPEGs store image data in 8x8 DCT tiles in the frequency domain. To
> restore an 8x8 tile in the image domain, an IDCT is computed, using up
> to 64 coefficients.
>
> If a JPEG with large dimensions is drawn at small scale on screen,
> pdfium used to completely decompress the JPEG to large size, doing a
> full IDCT for each tile, and then scale it down at decode time.
>
> Using scale_denom, pdfium can instead decode the image at 1/8th (or
> 1/4th, 1/2th) its size (in each dimension) by not doing the full IDCT,
> but instead only taking the one DC term (for 1/8th), or 2x2 or 4x4
> coefficents (for 1/4th, 1/2th).
>
> This not just saves work decoding the image, it also outputs a smaller
> image that is faster to downsample when painting it. For most images,
> this shouldn't look perceptively different than downsampling from the
> large bitmap.
>
> This kind of matches what Blink does when decoding JPEGs [1], and
> morally matches what pdfium already does for JPEG2000s after
> https://pdfium-review.googlesource.com/c/pdfium/+/99970.
>
> (Note that CPDF_ImageRenderer::StartLoadDIBBase() in
> cpdf_imagerenderer.cpp passes in the page size as max_size_required,
> which cpdf_dib.cpp uses to compute resolution_levels_to_skip. So image
> decoding currently isn't limited to the size the image is drawn at, but
> by the size of the bitmap the whole page is drawn at. This can be be
> improved later.)
>
> Depending on what data the encoder writes in padding rows / columns for
> images that aren't a multiple of the MCU size, the code might produce
> artifacts in the last row or column. So only do this for images with
> sizes a multiple of the MCU for now, see also https://crbug.com/890745
> and the discussion on
> https://github.com/libjpeg-turbo/libjpeg-turbo/issues/297. This is
> unfortunate, but currently there's no way to do better without hacks.
> (This also matches what Blink does.)
>
> Timing (hyperfine, --render-repeats): a 3072x2304 4:4:4 image drawn onto
> a 64pt page decodes in 40ms at scale_denom 8 vs 84ms at full resolution.
>
> 1:
> https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/platform/image-decoders/jpeg/jpeg_image_decoder.cc;l=430
> ...but evidently this is much harder to hit now then when that code was
> initially added, probably because JPEGs in Chrome now go down the
> DecodeYUV path, which doesn't set scale_denom (yet?).
>
> Depends on new baseline in
> https://pdfium-review.googlesource.com/c/pdfium_tests/+/151570
> and updates DEPS to pick it up.
>
> Plumb the existing resolution_levels_to_skip (computed in
> CPDF_DIB::StartLoadDIBBase from the render device size) into the DCT
> path. CreateDCTDecoder maps it to a libjpeg scale_denom of 1/2/4/8
> (capped at 8, the largest power-of-two DCT scaling libjpeg supports) and
> passes it to JpegModule::CreateDecoder. JpegDecoder sets
> cinfo.scale_denom before jpeg_start_decompress and reports the
> scaled-down output dimensions (ceil(dim/scale_denom), matching
> jpeg_core_output_dimensions). The DIB then adopts the decoder's reduced
> dimensions, mirroring the JPX path. libjpeg performs the reduction
> cheaply inside the IDCT (e.g. 1/8 uses a 1x1 IDCT), so far fewer pixels
> are produced for images shown much smaller than native.
>
> The calls to SetWidth()/SetHeight() are a slight behavior change for all
> JPEG data for invalid files: Previously, the code used width/height from
> the PDF dict, now it gets the size from the decoder. These sizes are
> supposed to match, but if they aren't, pdfium now uses the other one.
> This is arguably better, as that size dictates how much data is actually
> present.
>
> Bug: 531782505
> Change-Id: I66409898d028685813b274e063aedf52200be0dd
> Reviewed-on: https://pdfium-review.googlesource.com/c/pdfium/+/151550
> Commit-Queue: Nico Weber <thakis@chromium.org>
> Reviewed-by: Lei Zhang <thestig@chromium.org>
> Reviewed-by: Nico Weber <thakis@google.com>

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug: 531782505
Change-Id: If8fa628779381543e6243b75b0b07e26ae87ac12
Reviewed-on: https://pdfium-review.googlesource.com/c/pdfium/+/151870
Reviewed-by: Lei Zhang <thestig@chromium.org>
Commit-Queue: Lei Zhang <thestig@chromium.org>
Bot-Commit: rubber-stamper@appspot.gserviceaccount.com <rubber-stamper@appspot.gserviceaccount.com>
13 files changed
tree: 6cfb6746a5acaf3de45f58b6d9824860de623816
  1. build_overrides/
  2. constants/
  3. core/
  4. docs/
  5. fpdfsdk/
  6. fxbarcode/
  7. fxjs/
  8. infra/
  9. public/
  10. samples/
  11. skia/
  12. testing/
  13. third_party/
  14. tools/
  15. xfa/
  16. .clang-format
  17. .gitattributes
  18. .gitignore
  19. .gn
  20. .rustfmt.toml
  21. .style.yapf
  22. .vpython3
  23. AUTHORS
  24. BUILD.gn
  25. codereview.settings
  26. CONTRIBUTING.md
  27. DEPS
  28. DIR_METADATA
  29. LICENSE
  30. navbar.md
  31. OWNERS
  32. pdfium.gni
  33. PRESUBMIT.py
  34. PRESUBMIT_test.py
  35. PRESUBMIT_test_mocks.py
  36. README.md
  37. unsafe_buffers_paths.txt
README.md

PDFium

Prerequisites

PDFium uses the same build tooling as Chromium. See the platform-specific Chromium build instructions to get started, but replace Chromium's “Get the code” instructions with PDFium's.

CPU Architectures supported

The default architecture for Windows, Linux, and Mac is “x64”. On Windows, “x86” is also supported. GN parameter “target_cpu = "x86"” can be used to override the default value. If you specify Android build, the default CPU architecture will be “arm”.

It is expected that there are still some places lurking in the code which will not function properly on big-endian architectures. Bugs and/or patches are welcome, however providing this support is not a priority at this time.

Compilers supported

PDFium aims to be compliant with the Chromium policy.

Currently this means Clang. Former MSVC users should consider using clang-cl if needed. Community-contributed patches for gcc will be allowed. No MSVC patches will be taken.

Google employees

Run: download_from_google_storage --config and follow the authentication instructions. Note that you must authenticate with your @google.com credentials. Enter “0” if asked for a project-id.

Once you've done this, the toolchain will be installed automatically for you in the Generate the build files step below.

The toolchain will be in depot_tools\win_toolchain\vs_files\<hash>, and windbg can be found in depot_tools\win_toolchain\vs_files\<hash>\win_sdk\Debuggers.

If you want the IDE for debugging and editing, you will need to install it separately, but this is optional and not needed for building PDFium.

Get the code

The name of the top-level directory does not matter. In the following example, the directory name is “repo”. This directory must not have been used before by gclient config as each directory can only house a single gclient configuration.

mkdir repo
cd repo
gclient config --unmanaged https://pdfium.googlesource.com/pdfium.git
gclient sync
cd pdfium

On Linux, additional build dependencies need to be installed by running the following from the pdfium directory.

./build/install-build-deps.sh

Generate the build files

PDFium uses GN to generate the build files and Ninja to execute the build files. Both of these are included with the depot_tools checkout.

Selecting build configuration

PDFium may be built either with or without JavaScript support, and with or without XFA forms support. Both of these features are enabled by default. Also note that the XFA feature requires JavaScript.

Configuration is done by executing gn args <directory> to configure the build. This will launch an editor in which you can set the following arguments. By convention, <directory> should be named out/foo, and some tools / test support code only works if one follows this convention. A typical <directory> name is out/Debug.

use_remoteexec = false # Approved users only.  Do necessary setup & authentication first.
is_debug = true  # Enable debugging features.

# Set true to enable experimental Skia backend.
pdf_use_skia = false

# Set true to enable experimental Fontations backend.
pdf_enable_fontations = false

pdf_enable_xfa = true  # Set false to remove XFA support (implies JS support).
pdf_enable_v8 = true  # Set false to remove Javascript support.
is_component_build = false # Disable component build (Though it should work)

In order to minimize the build time for embedder applications, test executables like pdfium_test are not built by default. To build these, add the argument pdf_is_standalone = true.

By default, the entire project builds with C++20.

By default, PDFium expects to build with a clang compiler that provides additional chrome plugins. To build against a vanilla one lacking these, one must set clang_use_chrome_plugins = false.

When complete the arguments will be stored in <directory>/args.gn, and GN will automatically use the new arguments to generate build files. Should your files fail to generate, please double-check that you have set use_sysroot as indicated above.

Building the code

You can build the standalone test program by running: ninja -C <directory> pdfium_test You can build the entire product (which includes a few unit tests) by running: ninja -C <directory> pdfium_all

Running the standalone test program

The pdfium_test program supports reading, parsing, and rasterizing the pages of a .pdf file to .ppm or .png output image files (Windows supports two other formats). For example: <directory>/pdfium_test --ppm path/to/myfile.pdf. Note that this will write output images to path/to/myfile.pdf.<n>.ppm. Run pdfium_test --help to see all the options.

Testing

There are currently several test suites that can be run:

  • pdfium_unittests
  • pdfium_embeddertests
  • testing/tools/run_corpus_tests.py
  • testing/tools/run_javascript_tests.py
  • testing/tools/run_pixel_tests.py

It is possible the tests in the testing directory can fail due to font differences on the various platforms. These tests are reliable on the bots. If you see failures, it can be a good idea to run the tests on the tip-of-tree checkout to see if the same failures appear.

Pixel Tests

If your change affects rendering, a pixel test should be added. Simply add a .in or .pdf file in testing/resources/pixel and the pixel runner will pick it up at the next run.

Make sure that your test case doesn't have any copyright issues. It should also be a minimal test case focusing on the bug that renders the same way in many PDF viewers. Try to avoid binary data in streams by using the ASCIIHexDecode simply because it makes the PDF more readable in a text editor.

To try out your new test, you can call the run_pixel_tests.py script:

$ ./testing/tools/run_pixel_tests.py your_new_file.in

To generate the expected image, you can use the make_expected.sh script:

$ ./testing/tools/make_expected.sh your_new_file.pdf

Please make sure to have optipng installed which optimized the file size of the resulting png.

.in files

.in files are PDF template files. PDF files contain many byte offsets that have to be kept correct or the file won't be valid. The template makes this easier by replacing the byte offsets with certain keywords.

This saves space and also allows an easy way to reduce the test case to the essentials as you can simply remove everything that is not necessary.

A simple example can be found here.

To transform this into a PDF, you can use the fixup_pdf_template.py tool:

$ ./testing/tools/fixup_pdf_template.py your_file.in

This will create a your_file.pdf in the same directory as your_file.in.

There is no official style guide for the .in file, but a consistent style is preferred simply to help with readability. If possible, object numbers should be consecutive and /Type and /SubType should be on top of a dictionary to make object identification easier.

Embedding PDFium in your own projects

The public/ directory contains header files for the APIs available for use by embedders of PDFium. The PDFium project endeavors to keep these as stable as possible.

Outside of the public/ directory, code may change at any time, and embedders should not directly call these routines.

Code Coverage

Code coverage reports for PDFium can be generated in Linux development environments. Details can be found here.

Chromium provides code coverage reports for PDFium here. PDFium is located in third_party/pdfium in Chromium‘s source code. This includes code coverage from PDFium’s fuzzers.

Waterfall

The current health of the source tree can be found here.

Community

There are several mailing lists that are setup:

Note, the Reviews and Bugs lists are typically read-only.

Bugs

PDFium uses this bug tracker.

Report security bugs via Google Bughunters (Chrome VRP).

Project members and embedders can directly report security bugs using Chromium's security bug template.

Contributing code

See the CONTRIBUTING document for more information on contributing to the PDFium project.