OpenTimelineIO: Rearrange PyPI projects, break out adapters

Overview

The goal of this issue is to refactor the PyPI projects for OpenTimelineIO into two pypi projects from the current one:

  • OpenTimelineIO (the current package) becomes ONLY the core python wrapped library
    • Versioned whenever a new release of the core comes out from GitHub (similar to the current release process)
  • OpenTimelineIO-Plugins (a new package) is OpenTimelineIO + a curated set of plugins
    • Versioned whenever a plugin is added or removed from the list, or the OpenTimelineIO releases
    • A plugin may be removed from the list if it remains unsupported and falls behind the core, as determined by the OTIO TSC
    • PRs may be made to add or remove plugins from the list
    • The Plugins project will be in its own repository, with a README that describes briefly some criteria that the TSC is looking for when assessing whether to include a project or not
    • Plugins that are not present in OpenTimelineIO-Plugins may still be present on pip. Users may install extra plugins through pip that aren’t included in the OpenTimelineIO-Plugins project.

Outstanding tasks:

  • Remove the dependency on the contrib project from the top level setup.py
  • Add a note to the readme about the OpenTimelineIO-Plugins
  • Rename umbrella repository to OpenTimeilneIO-Plugins
  • Create pypi project for OpenTimelineIO-Plugins
  • add CI integration for pushing to pypi from Plugins repo
  • CI integration for checking new versions of OpenTimelineIO from the Plugins repo
  • register pypi packages for the adapter plugin projects <- at least for the ones that we plan on having the umbrella project include
  • break out adapters
    • as a TSC group we identify which adapters we want to include in the default installation
    • each adapter gets its own package, which can be individually installed
    • remove explicit references to the contrib package from the main library (see comment below) and documentation
    • remove the contrib directory from the repository
    • recommend a pinning strategy for how the adapters should handle their version dependencies + some CI for testing against new versions of the CORE
  • add version freezing to release script for parent projects
  • Add versioning strategy (noted below) to documentation (CONTRIBUTING.md or the main documentation)
  • Remove references to the adapters from the main documentation (or update them to point at the individual adapters, mention that they’re accessible via the -Plugins project)
  • CI for adapter projects need create new versions on manual activation rather than when tags are created
  • Show the plugin version in the otiopluginfo command output
  • Add a documentation page that describes project layout and how that relates PyPI package

Versioning strategy

  • OpenTimelineIO-Plugins project pins to a specific OpenTimelineIO project, corresponding to the current release, using the == constraint
  • Each adapter picks a versioning strategy against the OpenTimelineIOversion number
  • the -Plugins project depends on a set of adapters in a floating way, which lets the adapter repos individually deal with their dependency on the core
  • OpenTimelineIO-Plugins gets versioned only when plugins get added or removed, or the pinned version of OpenTimelineIO is updated

Why the OpenTimelineIO Does Not Include The Plugins ("OpenTimelineIO-core vs OpenTimelineIO-

Currently, if you pip install opentimelineio you get the contrib adapters + the core library. This suggestion switches that around such that pip install opentimelineio is only the core without the contrib adapters. We decided to make this shift for several reasons:

  • We want to make it clear to what degree the main project provides a warranty for components, and we want to emphasize the core format
  • We can update the version of the main OpenTimelineIO package without the plugins being updated if there is a compability issue
  • For software vendors, its more clear to identify the project version a software package uses or provides compatability for

Question

  • if we break an adapter from a change to OTIO core, how/when do we catch that? Do we [re]introduce smoke testing in the Plugins repo?

About this issue

  • Original URL
  • State: open
  • Created 2 years ago
  • Comments: 46 (20 by maintainers)

Most upvoted comments

With help from @JeanChristopheMorinPerso and @timlehr we were able to:

  • Publish all the adapter plugins as v1.0.0 to PyPI (maintainers can now use the “Create a new release” button in GitHub to release new versions to PyPI after bumping the version in the pyproject.toml)
  • Create a PR for the “batteries included” OpenTimelineIO-Plugins package that points to the OTIO core as well as the adapters

Next Steps:

  • Release OpenTimleineIO 0.16.0.dev1 to PyPI from the extract_adapters branch
  • Update OpenTimelineIO-Plugins to depend on opentimelineio==0.16.0.dev1 rather than the git link
  • Release OpenTimelineIO-Plugins 0.16.0.dev1 to PyPI

This should make it easy for users to test things out and give feedback.

extract adapters repos have been merged and repos have been transferred. otio-ale-adapter otio-hls-playlist-adapter

hls_playlist adapter repo is ready for transfer here

Also the PR to extract the adapter from core is ready but is blocked by the follow issue

In the meantime while this is being resolved, I’m going to start work on the ale plugin.

fcpx_xml adapter repo is ready: https://github.com/OpenTimelineIO/otio-fcpx-xml-adapter fcpx_xml adapter has been extracted from “extract_adapters” branch.

maya_sequencer adapter repo is ready: https://github.com/OpenTimelineIO/otio-maya-sequencer-adapter maya_sequencer adapter has been extracted from “extract_adapters” branch.

I’ll be working on moving the maya adapter.

You can start with the non-contrib ones, so we can demonstrate that this all works before deciding about each contrib adatper.

@mikemahony and I made this empty repo for experimentation. We can sketch this out and see how it fits before deciding on the final layout: https://github.com/OpenTimelineIO/otio-umbrella

I agree that having the repos and the pypi packages with clear names is important. The version history of a separate umbrella repo seems appealing too, but not essential.