Your First Pipeline

We are going to start very simply. Thus our very first pipeline only consists of three objects -- a Datasource, a Datasink, and a Dataframe.

To begin, we need to import the Datapipeline.py
Then we are going to create a function exanple0() which creates the necessary objects.

import Datapipeline as pipeline
def exanple0():
    aDataframe = pipeline.Dataframe()
    aDatasource = pipeline.Datasource()
    aDatasink = pipeline.Datasink()

In order to identify pipeline objects -- the building blocks of our pipeline - it is useful to give them a name. In addition, all pipeline objects also carry a numerical ID that is automatically created during the instantiation of the pipeline object.

def exanple0():
    aDataframe = pipeline.Dataframe()
    aDatasource = pipeline.Datasource()
    aDatasink = pipeline.Datasink()

    aDatasource.setName("Datasource")
    aDatasink.setName("Datasink")

Of course we can also print the names as well as the IDs of each of the objects.

def exanple0():
    aDataframe = pipeline.Dataframe()
    aDatasource = pipeline.Datasource()
    aDatasink = pipeline.Datasink()

    aDatasource.setName("Datasource")
    aDatasink.setName("Datasink")
    
    print(aDatasource.getName())
    print(aDatasink.getName())

def exanple0():
    aDataframe = pipeline.Dataframe()
    aDatasource = pipeline.Datasource()
    aDatasink = pipeline.Datasink()

    aDatasource.setName("Datasource")
    aDatasink.setName("Datasink")
    print(aDatasource.getName())
    print(aDatasource.getID())
    print(aDatasink.getName())
    print(aDatasink.getID())

With the basic blocks set up, we can now connect them to build our first pipeline. For that, we are connecting the output of the Datasource to the input of the Datasink.

def exanple0():
    aDataframe = pipeline.Dataframe()
    aDatasource = pipeline.Datasource()
    aDatasink = pipeline.Datasink()

    aDatasource.setName("Our Datasource")
    aDatasink.setName("Our Datasink")
    
    print(aDatasource.getName())
    print(aDatasource.getID())
    
    print(aDatasink.getName())
    print(aDatasink.getID())

    aDatasource.setOutput(aDatasink)

We are almost done! Our pipeline is set up and ready to run. All we need to do is to give it some data and then run it.

def exanple0():
    aDataframe = pipeline.Dataframe()
    aDatasource = pipeline.Datasource()
    aDatasink = pipeline.Datasink()

    aDatasource.setName("Our Datasource")
    aDatasink.setName("Our Datasink")
    
    print(aDatasource.getName())
    print(aDatasource.getID())
    
    print(aDatasink.getName())
    print(aDatasink.getID())
    
    aDatasource.setOutput(aDatasink)
    
    aDatasource.addDataframe(aDataframe)
    aDatasource.run()

Now let's run our pipeline and see what happens!
And we get something similar to this output:

Our Datasource
22247120
Our Datasink
21327534
Hello from:  Our Datasink  ID:  21327534
Frame number:  0

Note that the Datasink also says hello with its Name and ID. This simple feature is quite useful for debugging larger pipelines. Also, the Dataframe has a number. It is 0 because he has not set any. So let's fix this and rerun the pipeline.

def exanple0():
    aDataframe = pipeline.Dataframe()
    aDatasource = pipeline.Datasource()
    aDatasink = pipeline.Datasink()

    aDatasource.setName("Our Datasource")
    aDatasink.setName("Our Datasink")
    
    print(aDatasource.getName())
    print(aDatasource.getID())
    
    print(aDatasink.getName())
    print(aDatasink.getID())
    
    aDatasource.setOutput(aDatasink)
    aDataframe.setFrameNumber(1234)    
    aDatasource.addDataframe(aDataframe)
    aDatasource.run()

The output is now:

Our Datasource
46782035
Our Datasink
77057774
Hello from:  Our Datasink  ID:  77057774
Frame number:  1234

Congratulations!
You have developed and run your first working pipeline. Pat yourself on the shoulder and get yourself a cup of tea!