How To Install Python Dependencies For Dataflow

I have a very small python dataflow package, the structure of package looks like this . ├── __pycache__ ├── pubsubtobigq.py ├── requirements.txt └── venv t

Solution 1:

Dataflow workers cannot look into global dependencies https://cloud.google.com/dataflow/docs/resources/faq#how_do_i_handle_nameerrors

As suggested by quimiluzon@, try if your job works with DirectRunner. If so, this might work:

defparse_proto(message):
    from protobuf_to_dict import protobuf_to_dict
    dictinoary = protobuf_to_dict(message)