mlflow: databricks.proto error when importing MLflow after tensorflow
When I try to import MLflow in my project, I get the following error:
TypeError Traceback (most recent call last)
<ipython-input-1-d5dc4544d486> in <module>()
9 from keras.optimizers import SGD
10 from keras.preprocessing.image import ImageDataGenerator
---> 11 import mlflow
12
13 import numpy as np
/usr/local/lib/python3.5/dist-packages/mlflow/__init__.py in <module>()
6
7 # pylint: disable=wrong-import-position
----> 8 import mlflow.projects as projects # noqa
9 import mlflow.tracking as tracking # noqa
10
/usr/local/lib/python3.5/dist-packages/mlflow/projects.py in <module>()
16
17 from mlflow.entities.source_type import SourceType
---> 18 from mlflow.entities.param import Param
19 from mlflow import data
20 import mlflow.tracking as tracking
/usr/local/lib/python3.5/dist-packages/mlflow/entities/param.py in <module>()
1 from mlflow.entities._mlflow_object import _MLflowObject
----> 2 from mlflow.protos.service_pb2 import Param as ProtoParam
3
4
5 class Param(_MLflowObject):
/usr/local/lib/python3.5/dist-packages/mlflow/protos/service_pb2.py in <module>()
18
19 from mlflow.protos.scalapb import scalapb_pb2 as scalapb_dot_scalapb__pb2
---> 20 import mlflow.protos.databricks_pb2 as databricks__pb2
21
22
/usr/local/lib/python3.5/dist-packages/mlflow/protos/databricks_pb2.py in <module>()
25 serialized_pb=_b('\n\x10\x64\x61tabricks.proto\x1a google/protobuf/descriptor.proto\x1a\x15scalapb/scalapb.proto\"\x9a\x01\n\x14\x44\x61tabricksRpcOptions\x12 \n\tendpoints\x18\x01 \x03(\x0b\x32\r.HttpEndpoint\x12\x1f\n\nvisibility\x18\x02 \x01(\x0e\x32\x0b.Visibility\x12\x1f\n\x0b\x65rror_codes\x18\x03 \x03(\x0e\x32\n.ErrorCode\x12\x1e\n\nrate_limit\x18\x04 \x01(\x0b\x32\n.RateLimit\"N\n\x0cHttpEndpoint\x12\x14\n\x06method\x18\x01 \x01(\t:\x04POST\x12\x0c\n\x04path\x18\x02 \x01(\t\x12\x1a\n\x05since\x18\x03 \x01(\x0b\x32\x0b.ApiVersion\"*\n\nApiVersion\x12\r\n\x05major\x18\x01 \x01(\x05\x12\r\n\x05minor\x18\x02 \x01(\x05\"@\n\tRateLimit\x12\x11\n\tmax_burst\x18\x01 \x01(\x03\x12 \n\x18max_sustained_per_second\x18\x02 \x01(\x03\"\x8c\x01\n\x15\x44ocumentationMetadata\x12\x11\n\tdocstring\x18\x01 \x01(\t\x12\x10\n\x08lead_doc\x18\x02 \x01(\t\x12\x1f\n\nvisibility\x18\x03 \x01(\x0e\x32\x0b.Visibility\x12\x1b\n\x13original_proto_path\x18\x04 \x03(\t\x12\x10\n\x08position\x18\x05 \x01(\x05\"g\n\x1f\x44\x61tabricksServiceExceptionProto\x12\x1e\n\nerror_code\x18\x01 \x01(\x0e\x32\n.ErrorCode\x12\x0f\n\x07message\x18\x02 \x01(\t\x12\x13\n\x0bstack_trace\x18\x03 \x01(\t*?\n\nVisibility\x12\n\n\x06PUBLIC\x10\x01\x12\x0c\n\x08INTERNAL\x10\x02\x12\x17\n\x13PUBLIC_UNDOCUMENTED\x10\x03*\xf6\x04\n\tErrorCode\x12\x12\n\x0eINTERNAL_ERROR\x10\x01\x12\x1b\n\x17TEMPORARILY_UNAVAILABLE\x10\x02\x12\x0c\n\x08IO_ERROR\x10\x03\x12\x0f\n\x0b\x42\x41\x44_REQUEST\x10\x04\x12\x1c\n\x17INVALID_PARAMETER_VALUE\x10\xe8\x07\x12\x17\n\x12\x45NDPOINT_NOT_FOUND\x10\xe9\x07\x12\x16\n\x11MALFORMED_REQUEST\x10\xea\x07\x12\x12\n\rINVALID_STATE\x10\xeb\x07\x12\x16\n\x11PERMISSION_DENIED\x10\xec\x07\x12\x15\n\x10\x46\x45\x41TURE_DISABLED\x10\xed\x07\x12\x1a\n\x15\x43USTOMER_UNAUTHORIZED\x10\xee\x07\x12\x1b\n\x16REQUEST_LIMIT_EXCEEDED\x10\xef\x07\x12\x1d\n\x18INVALID_STATE_TRANSITION\x10\xd1\x0f\x12\x1b\n\x16\x43OULD_NOT_ACQUIRE_LOCK\x10\xd2\x0f\x12\x1c\n\x17RESOURCE_ALREADY_EXISTS\x10\xb9\x17\x12\x1c\n\x17RESOURCE_DOES_NOT_EXIST\x10\xba\x17\x12\x13\n\x0eQUOTA_EXCEEDED\x10\xa1\x1f\x12\x1c\n\x17MAX_BLOCK_SIZE_EXCEEDED\x10\xa2\x1f\x12\x1b\n\x16MAX_READ_SIZE_EXCEEDED\x10\xa3\x1f\x12\x13\n\x0e\x44RY_RUN_FAILED\x10\x89\'\x12\x1c\n\x17RESOURCE_LIMIT_EXCEEDED\x10\x8a\'\x12\x18\n\x13\x44IRECTORY_NOT_EMPTY\x10\xf1.\x12\x18\n\x13\x44IRECTORY_PROTECTED\x10\xf2.\x12\x1f\n\x1aMAX_NOTEBOOK_SIZE_EXCEEDED\x10\xf3.:@\n\nvisibility\x12\x1d.google.protobuf.FieldOptions\x18\xd0\x86\x03 \x01(\x0e\x32\x0b.Visibility::\n\x11validate_required\x12\x1d.google.protobuf.FieldOptions\x18\xd1\x86\x03 \x01(\x08:4\n\x0bjson_inline\x12\x1d.google.protobuf.FieldOptions\x18\xd2\x86\x03 \x01(\x08:1\n\x08json_map\x12\x1d.google.protobuf.FieldOptions\x18\xd3\x86\x03 \x01(\x08:J\n\tfield_doc\x12\x1d.google.protobuf.FieldOptions\x18\xd4\x86\x03 \x03(\x0b\x32\x16.DocumentationMetadata:D\n\x03rpc\x12\x1e.google.protobuf.MethodOptions\x18\xd0\x86\x03 \x01(\x0b\x32\x15.DatabricksRpcOptions:L\n\nmethod_doc\x12\x1e.google.protobuf.MethodOptions\x18\xd4\x86\x03 \x03(\x0b\x32\x16.DocumentationMetadata:N\n\x0bmessage_doc\x12\x1f.google.protobuf.MessageOptions\x18\xd4\x86\x03 \x03(\x0b\x32\x16.DocumentationMetadata:N\n\x0bservice_doc\x12\x1f.google.protobuf.ServiceOptions\x18\xd4\x86\x03 \x03(\x0b\x32\x16.DocumentationMetadata:H\n\x08\x65num_doc\x12\x1c.google.protobuf.EnumOptions\x18\xd4\x86\x03 \x03(\x0b\x32\x16.DocumentationMetadata:O\n\x15\x65num_value_visibility\x12!.google.protobuf.EnumValueOptions\x18\xd0\x86\x03 \x01(\x0e\x32\x0b.Visibility:S\n\x0e\x65num_value_doc\x12!.google.protobuf.EnumValueOptions\x18\xd4\x86\x03 \x03(\x0b\x32\x16.DocumentationMetadataB*\n#com.databricks.api.proto.databricks\xe2?\x02\x10\x01')
26 ,
---> 27 dependencies=[google_dot_protobuf_dot_descriptor__pb2.DESCRIPTOR,scalapb_dot_scalapb__pb2.DESCRIPTOR,])
28
29 _VISIBILITY = _descriptor.EnumDescriptor(
/usr/local/lib/python3.5/dist-packages/google/protobuf/descriptor.py in __new__(cls, name, package, options, serialized_pb, dependencies, public_dependencies, syntax, pool)
827 # TODO(amauryfa): use the pool passed as argument. This will work only
828 # for C++-implemented DescriptorPools.
--> 829 return _message.default_pool.AddSerializedFile(serialized_pb)
830 else:
831 return super(FileDescriptor, cls).__new__(cls)
TypeError: Couldn't build proto file into descriptor pool!
Invalid proto descriptor for file "databricks.proto":
databricks.proto: Import "scalapb/scalapb.proto" has not been loaded.
Any idea where the issue may come from? Thank you!
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Comments: 23 (11 by maintainers)
Commits related to this issue
- Merge pull request #33 from kevinykuo/feature/reorg-tracking-api Tracking API consistency — committed to rstudio/mlflow by kevinykuo 6 years ago
- Merge pull request #33 from mateiz/summit-news Add news item on SAIS talks and add Splice machine logo — committed to mlflow/mlflow by mateiz 5 years ago
- Merge pull request #33 from criteo-forks/sync-criteo-master Sync criteo master — committed to jdlesage/mlflow by t-henri 5 years ago
The root cause here is that
tensorflowis imported beforemlflow. When this happens, theprotobufmodule is first initialized bytensorflowwith the C implementation whereasmlflowexpectsprotobufto use the python implementation.The fix is to make
mlflowwork with the C implementation by dropping in the original.protofiles.