ufo2ft: Produces invalid CFF OTF files under Python 3 (repro available)

What I did: Compiled a simple UFO to OTF (CFF) using ufo2ft 2.5.0 (compileOTF(ufo, optimizeCFF=CFFOptimization.NONE))

What I expected: Resulting OTF file to be valid as per OTS, FontVal and macOS Font Book.

What actually happened: OTF file is unreadable by macOS Font Book and fails validation by OTS.

If I compile the same source with the same settings using the same version of ufo2ft using Python 2.7 instead of Python 3.7 the resulting OTF file is valid — this seems to indicate a bug related to Python 3.

Full repro available here: https://github.com/rsms/ufo2ft-py3-bug

Platform: macOS (Darwin 18.2.0 Darwin Kernel Version 18.2.0: Fri Oct 5 19:41:49 PDT 2018; root:xnu-4903.221.2~2/RELEASE_X86_64 x86_64)
Python 2: 2.7.15
Python 3: 3.7.2
ufo2ft: 2.5.0

About this issue

Original URL
State: closed
Created 5 years ago
Comments: 18 (13 by maintainers)

Commits related to this issue

upgrade ufo2ft that includes fixed fonttools. This means we can also drop our temporary fonttools patch. Related to https://github.com/fonttools/fonttools/pull/1430 and https://github.com/googlei18n/u... — committed to rsms/inter by rsms 5 years ago

Most upvoted comments

released ufo2ft 2.6.0

anthrotype on Jan 7, 2019

the culprit seems to be in the encodeFloat function of fontTools.misc.psCharStrings module (despite according to git blame it hasn’t changed in 17 years…)

https://github.com/fonttools/fonttools/blob/3ba285e2504c73cac4970ae846f56db2b41fa4a0/Lib/fontTools/misc/psCharStrings.py#L228

it’s calling str on the float before encoding it to binary, and on python3 the float repr seems to be more precise than on python2.

Your font UPEM is 2816 (which is not as common value as say 1000 or 2048, which perhaps explain why we haven’t caught this issue before). The FontMatrix value 1.0/2816, when it’s converted to str, becomes '0.000355113636364' in python 2.7 and '0.0003551136363636364' in python 3.7.

If I apply this patch to round the FontMatrix values to 15 decimal digits, the output from python2 and python3 is identical, and both your test fonts pass validation:

diff --git a/Lib/ufo2ft/outlineCompiler.py b/Lib/ufo2ft/outlineCompiler.py
index 60a4a97..965b575 100644
--- a/Lib/ufo2ft/outlineCompiler.py
+++ b/Lib/ufo2ft/outlineCompiler.py
@@ -983,7 +983,14 @@ class OutlineOTFCompiler(BaseOutlineCompiler):
         topDict.UnderlineThickness = otRound(underlineThickness)
         # populate font matrix
         unitsPerEm = otRound(getAttrWithFallback(info, "unitsPerEm"))
-        topDict.FontMatrix = [1.0 / unitsPerEm, 0, 0, 1.0 / unitsPerEm, 0, 0]
+        topDict.FontMatrix = [
+            round(1.0 / unitsPerEm, 15),
+            0,
+            0,
+            round(1.0 / unitsPerEm, 15),
+            0,
+            0,
+        ]
         # populate the width values
         if not any(hasattr(info, attr) and getattr(info, attr) is not None
                    for attr in ("postscriptDefaultWidthX",

I’m thinking of rather changing the encodeFloat function in upstream fonttools, to prevent this and similar issues with real numbers in CFF dicts.

In makeotf, they seem to round to 8 decimal digits https://github.com/adobe-type-tools/afdko/issues/174

I’ll sleep over it, and push a fix for it tomorrow.

anthrotype on Jan 3, 2019

thanks for filing this issue and providing an exact reproducer!

Comparing the ttx dumps of the two fonts generated respectively from python2 and python3 I only get this difference in the FontMatrix operator of the CFF table:

$ diff -Naur output{2,3}.ttx
--- output2.ttx	2019-01-03 21:40:37.000000000 +0000
+++ output3.ttx	2019-01-03 21:40:37.000000000 +0000
@@ -14,7 +14,7 @@
     <!-- Most of this table will be recalculated by the compiler -->
     <tableVersion value="1.0"/>
     <fontRevision value="1.0"/>
-    <checkSumAdjustment value="0x7def7972"/>
+    <checkSumAdjustment value="0x90ebc0da"/>
     <magicNumber value="0x5f0f3cf5"/>
     <flags value="00000000 00000011"/>
     <unitsPerEm value="2816"/>
@@ -197,7 +197,7 @@
       <UnderlineThickness value="220"/>
       <PaintType value="0"/>
       <CharstringType value="2"/>
-      <FontMatrix value="0.000355113636364 0 0 0.000355113636364 0 0"/>
+      <FontMatrix value="0.0003551136363636364 0 0 0.0003551136363636364 0 0"/>
       <FontBBox value="72 -660 1896 2708"/>
       <StrokeWidth value="0"/>
       <!-- charset is dumped separately as the 'GlyphOrder' element -->

the extra precision of the python3 float division seems to trigger this invalid font file somehow. I’ll investigate more tomorrow, but we probably need to make sure we round these floats to some reasonable number of digits.

https://github.com/googlei18n/ufo2ft/blob/39ab30c54ab71b24d96bb68e0965c41937d39b35/Lib/ufo2ft/outlineCompiler.py#L986

anthrotype on Jan 3, 2019

patch was merged upstream, a new fonttools release should be out later today, after which we shall bump the requirement in ufo2ft and release the latter as well.

anthrotype on Jan 7, 2019

by trial and error, it appears that what triggers this issue is the length (in bytes) of the encoded real number. When it is up to 10 bytes, it is accepted; if it is > 10, it is rejected as invalid. For example, the number -9.399999999999999, when run through encodeFloat (on py3) yields b'\x1e\xe9\xa3\x99\x99\x99\x99\x99\x99\x99\xff' which is 11 bytes long, and thus rejected. If I remove one digit or I remove the leading sign (I make the number positive with the same number of decimal digits), it is encoded with 10 bytes and it passes…

maybe the encodeFloat should be capped to max 10 bytes?

/cc @readroberts

anthrotype on Jan 4, 2019