I am using beam to read from gcp pubsub and writing into bigquery after some transformation. I am using colab to build my code. My code is running perfectly when i am running from colab whereas when I am using dataflow as runner i am getting NameError or Pickling error. I tried both save_main_session=True and save_main_session=False.
Appreciate your reply.