peeringdb: PeeringDB sync fails on MacOS

Sync is failing. It fails on IX 3233 which looks like a tunnel IX. Maybe #782 should be rethought?

Traceback (most recent call last):
  File "/usr/local/bin/peeringdb", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.8/site-packages/peeringdb/cli.py", line 62, in main
    return handler(config=cfg, **vars(options))
  File "/usr/local/lib/python3.8/site-packages/peeringdb/commands.py", line 20, in _wrapped
    r = func(*a, **k)
  File "/usr/local/lib/python3.8/site-packages/peeringdb/commands.py", line 219, in handle
    client.update_all(rs)
  File "/usr/local/lib/python3.8/site-packages/peeringdb/_update.py", line 66, in update_all
    self._atomic_update(lambda: ctx.sync_resource(r, since=since))
  File "/usr/local/lib/python3.8/site-packages/peeringdb/_update.py", line 78, in _atomic_update
    sync_func()
  File "/usr/local/lib/python3.8/site-packages/peeringdb/_update.py", line 66, in <lambda>
    self._atomic_update(lambda: ctx.sync_resource(r, since=since))
  File "/usr/local/lib/python3.8/site-packages/peeringdb/_tasks_async.py", line 65, in _wrapped
    return loop.run_until_complete(func(*a, **k))
  File "/usr/local/Cellar/python@3.8/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/peeringdb/_tasks_async.py", line 41, in _wrapped
    item = gen.send(r)
  File "/usr/local/lib/python3.8/site-packages/peeringdb/_update.py", line 287, in sync_row
    B.clean(obj)
  File "/usr/local/lib/python3.8/site-packages/django_peeringdb/client_adaptor/backend.py", line 150, in clean
    obj.full_clean()
  File "/usr/local/lib/python3.8/site-packages/django/db/models/base.py", line 1171, in full_clean
    raise ValidationError(errors)
django.core.exceptions.ValidationError: {'ix': ['Internet Exchange instance with id 3233 does not exist.']}
%

About this issue

Original URL
State: closed
Created 4 years ago
Reactions: 3
Comments: 33 (19 by maintainers)

Most upvoted comments

I’ll point out that my instance doesn’t fail with a “virtual” IXP, and in the network objects. Without digging into the code, I think we can safely rule out that as a possible cause.

mcmanuss8 on Aug 3, 2020

django.core.exceptions.ValidationError: {'ix': ['Internet Exchange instance with id 1522 does not exist.']}

deleting the db, and starting again, i get error on a new network/ix/fac. i’m going to spin up a VM now to test.

tpaseka on Aug 3, 2020

I built and installed on MacOS (Catalina 10.15.5). Ran into a different (but similar) issue with a network not existing, that clearly does: https://www.peeringdb.com/net/2425

% /usr/local/bin/pip3 freeze | grep -i django
Django==2.2.9
django-countries==6.1.2
django-handleref==0.5.0
django-inet==0.4.0
django-peeringdb==2.2.0
% /usr/local/bin/pip3 freeze | grep -i peering
django-peeringdb==2.2.0
peeringdb==1.0.0
% head -n 1 /usr/local/bin/peeringdb
#!/usr/local/opt/python/bin/python3.7
% /usr/local/opt/python/bin/python3.7 --version
Python 3.7.6
% /usr/local/bin/peeringdb sync
Syncing to https://www.peeringdb.com/api
Updating resources: org fac net ix ixfac ixlan ixpfx netfac netixlan poc
Fetching & updating all: org
Updates to be processed: 19175
Ignoring object updated after sync began: (org-19478)
Ignoring object updated after sync began: (org-26881)
Fetching & updating all: fac
Updates to be processed: 3795
Fetching & updating all: net
Updates to be processed: 19673
Ignoring object updated after sync began: (net-2425)
Ignoring object updated after sync began: (net-10335)
Ignoring object updated after sync began: (net-14796)
Ignoring object updated after sync began: (net-15745)
Ignoring object updated after sync began: (net-16245)
Ignoring object updated after sync began: (net-19151)
Ignoring object updated after sync began: (net-23047)
Ignoring object updated after sync began: (net-24132)
Ignoring object updated after sync began: (net-24133)
Ignoring object updated after sync began: (net-24137)
Fetching & updating all: ix
Updates to be processed: 815
Ignoring object updated after sync began: (ix-1896)
Fetching & updating all: ixfac
Updates to be processed: 2188
Fetching & updating all: ixlan
Updates to be processed: 814
Ignoring object updated after sync began: (ixlan-1896)
Fetching & updating all: ixpfx
Updates to be processed: 1566
Ignoring object updated after sync began: (ixpfx-2463)
Ignoring object updated after sync began: (ixpfx-2464)
Fetching & updating all: netfac
Updates to be processed: 31017
Ignoring object updated after sync began: (netfac-43093)
Ignoring object updated after sync began: (netfac-43094)
Ignoring object updated after sync began: (netfac-43092)
Traceback (most recent call last):
  File "/usr/local/bin/peeringdb", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.7/site-packages/peeringdb/cli.py", line 62, in main
    return handler(config=cfg, **vars(options))
  File "/usr/local/lib/python3.7/site-packages/peeringdb/commands.py", line 20, in _wrapped
    r = func(*a, **k)
  File "/usr/local/lib/python3.7/site-packages/peeringdb/commands.py", line 219, in handle
    client.update_all(rs)
  File "/usr/local/lib/python3.7/site-packages/peeringdb/_update.py", line 66, in update_all
    self._atomic_update(lambda: ctx.sync_resource(r, since=since))
  File "/usr/local/lib/python3.7/site-packages/peeringdb/_update.py", line 78, in _atomic_update
    sync_func()
  File "/usr/local/lib/python3.7/site-packages/peeringdb/_update.py", line 66, in <lambda>
    self._atomic_update(lambda: ctx.sync_resource(r, since=since))
  File "/usr/local/lib/python3.7/site-packages/peeringdb/_tasks_async.py", line 65, in _wrapped
    return loop.run_until_complete(func(*a, **k))
  File "/usr/local/Cellar/python/3.7.6_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/base_events.py", line 583, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.7/site-packages/peeringdb/_tasks_async.py", line 41, in _wrapped
    item = gen.send(r)
  File "/usr/local/lib/python3.7/site-packages/peeringdb/_update.py", line 287, in sync_row
    B.clean(obj)
  File "/usr/local/lib/python3.7/site-packages/django_peeringdb/client_adaptor/backend.py", line 150, in clean
    obj.full_clean()
  File "/usr/local/lib/python3.7/site-packages/django/db/models/base.py", line 1203, in full_clean
    raise ValidationError(errors)
django.core.exceptions.ValidationError: {'net': ['Network instance with id 2425 does not exist.']}

And a subsequent run:
% /usr/local/bin/peeringdb sync
Syncing to https://www.peeringdb.com/api
Updating resources: org fac net ix ixfac ixlan ixpfx netfac netixlan poc
Fetching & updating all: org
Updates to be processed: 2
Ignoring object updated after sync began: (org-19478)
Ignoring object updated after sync began: (org-26881)
Fetching & updating all: fac
Updates to be processed: 0
Fetching & updating all: net
Updates to be processed: 11
Ignoring object updated after sync began: (net-16245)
Ignoring object updated after sync began: (net-15745)
Ignoring object updated after sync began: (net-23047)
Ignoring object updated after sync began: (net-19151)
Ignoring object updated after sync began: (net-14796)
Ignoring object updated after sync began: (net-10335)
Ignoring object updated after sync began: (net-24132)
Ignoring object updated after sync began: (net-24133)
Ignoring object updated after sync began: (net-24137)
Ignoring object updated after sync began: (net-21889)
Fetching & updating all: ix
Updates to be processed: 1
Ignoring object updated after sync began: (ix-1896)
Fetching & updating all: ixfac
Updates to be processed: 0
Fetching & updating all: ixlan
Updates to be processed: 1
Ignoring object updated after sync began: (ixlan-1896)
Fetching & updating all: ixpfx
Updates to be processed: 2
Ignoring object updated after sync began: (ixpfx-2463)
Ignoring object updated after sync began: (ixpfx-2464)
Fetching & updating all: netfac
Updates to be processed: 31017
Ignoring object updated after sync began: (netfac-43093)
Ignoring object updated after sync began: (netfac-43094)
Ignoring object updated after sync began: (netfac-43092)
Traceback (most recent call last):
  File "/usr/local/bin/peeringdb", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.7/site-packages/peeringdb/cli.py", line 62, in main
    return handler(config=cfg, **vars(options))
  File "/usr/local/lib/python3.7/site-packages/peeringdb/commands.py", line 20, in _wrapped
    r = func(*a, **k)
  File "/usr/local/lib/python3.7/site-packages/peeringdb/commands.py", line 219, in handle
    client.update_all(rs)
  File "/usr/local/lib/python3.7/site-packages/peeringdb/_update.py", line 66, in update_all
    self._atomic_update(lambda: ctx.sync_resource(r, since=since))
  File "/usr/local/lib/python3.7/site-packages/peeringdb/_update.py", line 78, in _atomic_update
    sync_func()
  File "/usr/local/lib/python3.7/site-packages/peeringdb/_update.py", line 66, in <lambda>
    self._atomic_update(lambda: ctx.sync_resource(r, since=since))
  File "/usr/local/lib/python3.7/site-packages/peeringdb/_tasks_async.py", line 65, in _wrapped
    return loop.run_until_complete(func(*a, **k))
  File "/usr/local/Cellar/python/3.7.6_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/base_events.py", line 583, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.7/site-packages/peeringdb/_tasks_async.py", line 41, in _wrapped
    item = gen.send(r)
  File "/usr/local/lib/python3.7/site-packages/peeringdb/_update.py", line 287, in sync_row
    B.clean(obj)
  File "/usr/local/lib/python3.7/site-packages/django_peeringdb/client_adaptor/backend.py", line 150, in clean
    obj.full_clean()
  File "/usr/local/lib/python3.7/site-packages/django/db/models/base.py", line 1203, in full_clean
    raise ValidationError(errors)
django.core.exceptions.ValidationError: {'net': ['Network instance with id 15745 does not exist.']}

mcmanuss8 on Aug 3, 2020

% python3 -V
Python 3.8.5

OSX 10.15.6

tpaseka on Jul 31, 2020

That WAS a fresh table.

mahtin on Jul 28, 2020