nteract: Rendering of "application/vnd.dataresource+json" broken

Based in discussion in the weekly meeting, the todos for the data-explorer package here are:

• Make primaryKey an optional property

  • Either by generating a default range index enumerating rows, or some other method

• Potentially add some schema validation that provides a useful error message if there is a problem.

Great, thanks a lot!

I’m going to try to repro and look into this morning, here is my hypothesis:

I can tell from above that IJulia is returning a bundle with the proper dataresource key, but nteract/data-explorer can’t parse the data. either, 1.) IJulia is not providing the proper data resource format of the data set. or 2.) data-explorer is not handling the data properly (maybe treating an optional key as required)

Here is how we type the data payload in data-explorer based on the data resource spec:

interface DataProps {
  schema: Schema;
  data: [fieldName: string]: number | string;
}

export interface Schema {
  fields: Field[];
  pandas_version: string;
  primaryKey: Array<Field["name"]>;
}

export interface Field {
  name: string;
  //  types provided by the kernel, literal strings:
  type: "integer" | "datetime" | "number" | "string" | "boolean";
}

Example from the iris dataset:

"schema": {
    "fields": [
      { "name": "index", "type": "integer" },
      { "name": "sepal_length", "type": "number" },
      { "name": "sepal_width", "type": "number" },
      { "name": "petal_length", "type": "number" },
      { "name": "petal_width", "type": "number" },
      { "name": "species", "type": "string" }
    ],
    "primaryKey": ["index"],
    "pandas_version": "0.20.0"
  },
"data": [
    {
      "index": 0,
      "sepal_length": 5.1,
      "sepal_width": 3.5,
      "petal_length": 1.4,
      "petal_width": 0.2,
      "species": "setosa"
    },
    {
      "index": 1,
      "sepal_length": 4.9,
      "sepal_width": 3,
      "petal_length": 1.4,
      "petal_width": 0.2,
      "species": "setosa"
    },
    // ...

Once we get out from under our typescript conversion muck, I want to take a full look at this. 😬

Thanks @BenRussert for taking a look at this.

Ah, yes, if I edit the file manually and add that, it works!

But, looking at the spec here (and in particular the “Primary Key” section), it seems pretty clear that this field is optional, right? I also wouldn’t know what to put there: many tables don’t have a primary key, and adding a column that just has row numbers seems also undesirable, because then the grid would show a column that doesn’t even exist in the source data, right?

It seems to me that right thing to do here is to make data-explorer be able to handle cases where there is no primaryKey field. I think it actually did in the past, because there was a time when all of this worked, and I didn’t change anything in the meantime on the julia side.