fonttools: hb.repack failed to serialize 'GPOS', works with pure fonttools

I’m trying to build a variable font, and it fails with hb.repack failed to serialize 'GPOS' when uharfbuzz is installed, but works when uharfbuzz is not available (but then fontbakery fails …).

I’ve seen some similar issues which were apparently resolved, so I’m wondering if this still happening is a bug?

FontTools’ successful fixing:

INFO:fontTools.ttLib.tables.otBase:Attempting to fix OTLOffsetOverflowError ('GPOS', 'LookupIndex:', 3, 'SubTableIndex:', 0, 'ItemName:', 'PairSet.<none>', 'ItemIndex:', None)
INFO:fontTools.ttLib.tables.otBase:Attempting to fix OTLOffsetOverflowError ('GPOS', 'LookupIndex:', 3, 'SubTableIndex:', 0, 'ItemName:', 'PairSet.<none>', 'ItemIndex:', None)
INFO:fontTools.ttLib.tables.otBase:Attempting to fix OTLOffsetOverflowError ('GPOS', 'LookupIndex:', 3, 'SubTableIndex:', 0, 'ItemName:', 'PairSet.<none>', 'ItemIndex:', None)
INFO:fontTools.ttLib.tables.otBase:Attempting to fix OTLOffsetOverflowError ('GPOS', 'LookupIndex:', 3, 'SubTableIndex:', 0, 'ItemName:', 'PairSet.<none>', 'ItemIndex:', None)
INFO:fontTools.ttLib.tables.otBase:Attempting to fix OTLOffsetOverflowError ('GPOS', 'LookupIndex:', 3, 'SubTableIndex:', 0, 'ItemName:', 'PairSet.<none>', 'ItemIndex:', None)
INFO:fontTools.ttLib.tables.otBase:Attempting to fix OTLOffsetOverflowError ('GPOS', 'LookupIndex:', 3, 'SubTableIndex:', 0, 'ItemName:', 'PairSet.<none>', 'ItemIndex:', None)
INFO:fontTools.ttLib.tables.otBase:Attempting to fix OTLOffsetOverflowError ('GPOS', 'LookupIndex:', 3, 'SubTableIndex:', 0, 'ItemName:', 'PairSet.<none>', 'ItemIndex:', None)
INFO:fontTools.ttLib.tables.otBase:Attempting to fix OTLOffsetOverflowError ('GPOS', 'LookupIndex:', 3, 'SubTableIndex:', 0, 'ItemName:', 'PairSet.<none>', 'ItemIndex:', None)
INFO:fontTools.ttLib.tables.otBase:Attempting to fix OTLOffsetOverflowError ('GPOS', 'LookupIndex:', 3, 'SubTableIndex:', 0, 'ItemName:', 'PairSet.<none>', 'ItemIndex:', None)
INFO:fontTools.ttLib.tables.otBase:Attempting to fix OTLOffsetOverflowError ('GPOS', 'LookupIndex:', 3, 'SubTableIndex:', 0, 'ItemName:', 'PairSet.<none>', 'ItemIndex:', None)
INFO:fontTools.ttLib.tables.otBase:Attempting to fix OTLOffsetOverflowError ('GPOS', 'LookupIndex:', 3, 'SubTableIndex:', 9, 'ItemName:', 'Coverage', 'ItemIndex:', None)
INFO:fontTools.ttLib.tables.otBase:Attempting to fix OTLOffsetOverflowError ('GPOS', 'LookupIndex:', 3, 'SubTableIndex:', 9, 'ItemName:', 'Coverage', 'ItemIndex:', None)
INFO:fontTools.ttLib.tables.otBase:Attempting to fix OTLOffsetOverflowError ('GPOS', 'LookupIndex:', 3, 'SubTableIndex:', 9, 'ItemName:', '<none>', 'ItemIndex:', None)

Traceback with harfbuzz:

DEBUG:fontTools.ttLib.tables.otBase:serializing 'GPOS' with hb.repack
WARNING:fontTools.ttLib.tables.otBase:hb.repack failed to serialize 'GPOS', reverting to pure-python serializer; the error message was: RepackerError
INFO:fontTools.ttLib.tables.otBase:Attempting to fix OTLOffsetOverflowError ('GPOS', 'LookupIndex:', 3, 'SubTableIndex:', 0, 'ItemName:', 'PairSet.<none>', 'ItemIndex:', None)
DEBUG:fontTools.ttLib.tables.otBase:serializing 'GPOS' with hb.repack
INFO:fontTools.ttLib.tables.otBase:Attempting to fix OTLOffsetOverflowError ('GPOS', 'LookupIndex:', 3, 'SubTableIndex:', 0, 'ItemName:', 'PairSet.<none>', 'ItemIndex:', None)
DEBUG:fontTools.ttLib.tables.otBase:serializing 'GPOS' with hb.repack
INFO:fontTools.ttLib.tables.otBase:Attempting to fix OTLOffsetOverflowError ('GPOS', 'LookupIndex:', 3, 'SubTableIndex:', 0, 'ItemName:', 'PairSet.<none>', 'ItemIndex:', None)
DEBUG:fontTools.ttLib.tables.otBase:serializing 'GPOS' with hb.repack
INFO:fontTools.ttLib.tables.otBase:Attempting to fix OTLOffsetOverflowError ('GPOS', 'LookupIndex:', 3, 'SubTableIndex:', 0, 'ItemName:', 'PairSet.<none>', 'ItemIndex:', None)
DEBUG:fontTools.ttLib.tables.otBase:serializing 'GPOS' with hb.repack
INFO:fontTools.ttLib.tables.otBase:Attempting to fix OTLOffsetOverflowError ('GPOS', 'LookupIndex:', 3, 'SubTableIndex:', 0, 'ItemName:', 'PairSet.<none>', 'ItemIndex:', None)
DEBUG:fontTools.ttLib.tables.otBase:serializing 'GPOS' with hb.repack
INFO:fontTools.ttLib.tables.otBase:Attempting to fix OTLOffsetOverflowError ('GPOS', 'LookupIndex:', 3, 'SubTableIndex:', 0, 'ItemName:', 'PairSet.<none>', 'ItemIndex:', None)
DEBUG:fontTools.ttLib.tables.otBase:serializing 'GPOS' with hb.repack
INFO:fontTools.ttLib.tables.otBase:Attempting to fix OTLOffsetOverflowError ('GPOS', 'LookupIndex:', 3, 'SubTableIndex:', 0, 'ItemName:', 'PairSet.<none>', 'ItemIndex:', None)
DEBUG:fontTools.ttLib.tables.otBase:serializing 'GPOS' with hb.repack
INFO:fontTools.ttLib.tables.otBase:Attempting to fix OTLOffsetOverflowError ('GPOS', 'LookupIndex:', 3, 'SubTableIndex:', 0, 'ItemName:', 'PairSet.<none>', 'ItemIndex:', None)
DEBUG:fontTools.ttLib.tables.otBase:serializing 'GPOS' with hb.repack
INFO:fontTools.ttLib.tables.otBase:Attempting to fix OTLOffsetOverflowError ('GPOS', 'LookupIndex:', 3, 'SubTableIndex:', 0, 'ItemName:', 'PairSet.<none>', 'ItemIndex:', None)
DEBUG:fontTools.ttLib.tables.otBase:serializing 'GPOS' with hb.repack
INFO:fontTools.ttLib.tables.otBase:Attempting to fix OTLOffsetOverflowError ('GPOS', 'LookupIndex:', 3, 'SubTableIndex:', 0, 'ItemName:', 'PairSet.<none>', 'ItemIndex:', None)
DEBUG:fontTools.ttLib.tables.otBase:serializing 'GPOS' with hb.repack
INFO:fontTools.ttLib.tables.otBase:Attempting to fix OTLOffsetOverflowError ('GPOS', 'LookupIndex:', 3, 'SubTableIndex:', 0, 'ItemName:', 'PairSet.<none>', 'ItemIndex:', None)
DEBUG:fontTools.ttLib.tables.otBase:serializing 'GPOS' with hb.repack
INFO:fontTools.ttLib.tables.otBase:Attempting to fix OTLOffsetOverflowError ('GPOS', 'LookupIndex:', 3, 'SubTableIndex:', 0, 'ItemName:', 'PairSet.<none>', 'ItemIndex:', None)
DEBUG:fontTools.ttLib.tables.otBase:serializing 'GPOS' with hb.repack
INFO:fontTools.ttLib.tables.otBase:Attempting to fix OTLOffsetOverflowError ('GPOS', 'LookupIndex:', 3, 'SubTableIndex:', 0, 'ItemName:', 'PairSet.<none>', 'ItemIndex:', None)
DEBUG:fontTools.ttLib.tables.otBase:serializing 'GPOS' with hb.repack
INFO:fontTools.ttLib.tables.otBase:Attempting to fix OTLOffsetOverflowError ('GPOS', 'LookupIndex:', 3, 'SubTableIndex:', 0, 'ItemName:', 'PairSet.<none>', 'ItemIndex:', None)
DEBUG:fontTools.ttLib.tables.otBase:serializing 'GPOS' with hb.repack
INFO:fontTools.ttLib.tables.otBase:Attempting to fix OTLOffsetOverflowError ('GPOS', 'LookupIndex:', 3, 'SubTableIndex:', 0, 'ItemName:', 'PairSet.<none>', 'ItemIndex:', None)
ERROR:root:In 'master_ufo/TestVariable.designspace': Generating fonts from Designspace failed: ('GPOS', 'LookupIndex:', 3, 'SubTableIndex:', 0, 'ItemName:', 'PairSet.<none>', 'ItemIndex:', None)
Traceback (most recent call last):
  File "/Users/jens/Code/fonttools/Lib/fontTools/ttLib/tables/otBase.py", line 111, in compile
    return writer.getAllDataUsingHarfbuzz()
  File "/Users/jens/Code/fonttools/Lib/fontTools/ttLib/tables/otBase.py", line 555, in getAllDataUsingHarfbuzz
    return hb.repack(data, obj_list)
  File "src/uharfbuzz/_harfbuzz.pyx", line 1219, in uharfbuzz._harfbuzz.repack
uharfbuzz._harfbuzz.RepackerError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/jens/Code/fonttools/Lib/fontTools/ttLib/tables/otBase.py", line 343, in getData
    items[i] = packUShort(item.pos - pos)
  File "/Users/jens/Code/fonttools/Lib/fontTools/ttLib/tables/otBase.py", line 730, in packUShort
    return struct.pack(">H", value)
struct.error: 'H' format requires 0 <= number <= 65535

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/jens/Code/fontmake/Lib/fontmake/font_project.py", line 1057, in run_from_designspace
    self._run_from_designspace_interpolatable(
  File "/Users/jens/Code/fontmake/Lib/fontmake/font_project.py", line 1147, in _run_from_designspace_interpolatable
    self.build_variable_fonts(
  File "/Users/jens/Code/fontmake/Lib/fontmake/font_project.py", line 378, in build_variable_fonts
    fonts = ufo2ft.compileVariableTTFs(
  File "/Users/jens/Documents/Schriften/Variable-Fonts/venv/lib/python3.10/site-packages/ufo2ft/__init__.py", line 605, in compileVariableTTFs
    vfNameToTTFont[vfName] = call_postprocessor(
  File "/Users/jens/Documents/Schriften/Variable-Fonts/venv/lib/python3.10/site-packages/ufo2ft/__init__.py", line 77, in call_postprocessor
    postProcessor = postProcessorClass(otf, ufo, glyphSet=glyphSet)
  File "/Users/jens/Documents/Schriften/Variable-Fonts/venv/lib/python3.10/site-packages/ufo2ft/postProcessor.py", line 50, in __init__
    self.otf = _reloadFont(otf)
  File "/Users/jens/Documents/Schriften/Variable-Fonts/venv/lib/python3.10/site-packages/ufo2ft/postProcessor.py", line 396, in _reloadFont
    font.save(stream)
  File "/Users/jens/Code/fonttools/Lib/fontTools/ttLib/ttFont.py", line 184, in save
    writer_reordersTables = self._save(tmp)
  File "/Users/jens/Code/fonttools/Lib/fontTools/ttLib/ttFont.py", line 224, in _save
    self._writeTable(tag, writer, done, tableCache)
  File "/Users/jens/Code/fonttools/Lib/fontTools/ttLib/ttFont.py", line 647, in _writeTable
    tabledata = self.getTableData(tag)
  File "/Users/jens/Code/fonttools/Lib/fontTools/ttLib/ttFont.py", line 669, in getTableData
    return self.tables[tag].compile(self)
  File "/Users/jens/Code/fonttools/Lib/fontTools/ttLib/tables/otBase.py", line 128, in compile
    return writer.getAllData(remove_duplicate=False)
  File "/Users/jens/Code/fonttools/Lib/fontTools/ttLib/tables/otBase.py", line 585, in getAllData
    tableData = table.getData()
  File "/Users/jens/Code/fonttools/Lib/fontTools/ttLib/tables/otBase.py", line 348, in getData
    raise OTLOffsetOverflowError(overflowErrorRecord)
fontTools.ttLib.tables.otBase.OTLOffsetOverflowError: ('GPOS', 'LookupIndex:', 3, 'SubTableIndex:', 0, 'ItemName:', 'PairSet.<none>', 'ItemIndex:', None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/jens/Code/fontmake/Lib/fontmake/__main__.py", line 627, in main
    project.run_from_designspace(inputs.designspace_path, **args)
  File "/Users/jens/Code/fontmake/Lib/fontmake/font_project.py", line 1072, in run_from_designspace
    raise FontmakeError(
fontmake.errors.FontmakeError: In 'master_ufo/TestVariable.designspace': Generating fonts from Designspace failed: ('GPOS', 'LookupIndex:', 3, 'SubTableIndex:', 0, 'ItemName:', 'PairSet.<none>', 'ItemIndex:', None)
make: *** [variable_ttf/TestVariable.ttf] Error 1

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 25 (15 by maintainers)

Commits related to this issue

[otBase] temporarily disable extension sharing during harfbuzz repacking If extension sharing is enabled some fonts may fail to repack due to fontTools repacker getting stuck in a unrecoverable error... — committed to fonttools/fonttools by anthrotype 2 years ago

Most upvoted comments

It looks like this fix is also needed: https://github.com/harfbuzz/harfbuzz/commit/9559d3c1c16812ed202852977ab11f10e3ebe804

I’ll ask to get a new harfbuzz and uharfbuzz release made.

garretrieger on Oct 12, 2022

uharfbuzz 0.30.0 still does not yet have support for table splitting of MarkBasePos, I’m working to get a new release cut which will include support for MarkBasePos splitting and some additional bugfixes: https://github.com/harfbuzz/uharfbuzz/issues/136

I tested building the provided font against uharfbuzz and harfbuzz head and it works.

garretrieger on Oct 7, 2022

Yes that’s right. When extension sharing is enabled it also enables sharing between extensions during the fallback to fontTools packing. In some cases this can lead fontTools to hit one of the cases where it gives up and bails out of repacking. Even though it could still make progress by splitting lookups. The immediate fix is to revert the changes from #2626. I’m looking into how to restructure the fallback from harfbuzz to fontTools to prevent this issue while allowing the harfbuzz side to keep sharing between extensions.

garretrieger on Jun 23, 2022