Our current pipeline produces a stream of random numbers.
To further process the data, we need a plugin that accesses the previously generated data, does something with it, and then attaches the result back to the data frame. In order to write our DataProcessingPlugin, we make use of Object-Oriented-Programming OOP. Thus, we don't have to write already existing code and only add additional functionality. In OOP terminology this means that we are going to inherit the exiting functionality from pipeline.Plugin which we will generally refer to as the superclass of our plugins
For our own plugin, all we need to do is adding a tiny bit of extra code that processes the data the way we want to.
Remember, so far we have only directly used pipeline. Plugin. And all it does is buffering data, say hello to us, and pass on the data.
Our goal now is to add some extra code that does something with data before it passed on. So, let's do this!
We start with the class DataProcessingPlugin which processes the data.
class DataProcessingPlugin(pipeline. Plugin): #Here we define our class that inherits functionality from the class pipeline.Plugin.
def __init__(self):
super().__init__() #our init function just calls __init__() from pipeline.Plugin
def process(self): #All plugins do data processing by calling their process() method.
super().process() #Call the process method from the superclass pipeline.Plugin
randomData = super().getDataframe().getData() #Here we get the previously attached data
randomData *= 1000 #Multiply the random data with 1000
super().getDataframe().setData(randomData) #Attach the processed data back to the Dataframe Nice! Let's move on to implementing the DataVisualizationPlugin to visualize our data by printing it on the screen.
The code is almost identical to the one of the DataProcessingPlugin. The big differences are that the data is printed and that the data in the Dataframe is not modified.
class DataVisualizationPlugin(pipeline.Plugin):
def __init__(self):
super().__init__()
def process(self):
super().process()
randomData = super().getDataframe().getData()
print("The data value is: ",randomData)We are now ready to use our own plugins in our own pipeline!
import numpy as np
def example4():
aDatasource = pipeline.Datasource()
aDatasink = pipeline.Datasink()
aDataProcessingPlugin = DataProcessingPlugin()
aDataVisualizationPlugin = DataVisualizationPlugin()
aDatasource.setName("Our Datasource")
aDatasink.setName("Our Datasink")
aDataProcessingPlugin.setName("Our DataProcessingPlugin")
aDataVisualizationPlugin.setName("Our DataVisualizationPlugin")
aDatasource.setOutput(aDataProcessingPlugin)
aDataProcessingPlugin.setOutput(aDataVisualizationPlugin)
aDataVisualizationPlugin.setOutput(aDatasink)
for n in range(100):
aDataframe = pipeline.Dataframe()
aDataframe.setFrameNumber(n)
aDataframe.setData(np.random.random(1))
aDatasource.addDataframe(aDataframe)
aDatasource.run()If we run this code we get:
Frame number: 96
Hello from: Our DataProcessingPlugin ID: 2025292
Frame number: 97
Hello from: Our DataVisualizationPlugin ID: 54597490
Frame number: 97
The data value is: [457.8713245]
Hello from: Our Datasink ID: 68123707
Frame number: 97
Hello from: Our DataProcessingPlugin ID: 2025292
Frame number: 98
Hello from: Our DataVisualizationPlugin ID: 54597490
Frame number: 98
The data value is: [539.01199379]
Hello from: Our Datasink ID: 68123707
Frame number: 98
Hello from: Our DataProcessingPlugin ID: 2025292
Frame number: 99
Hello from: Our DataVisualizationPlugin ID: 54597490
Frame number: 99
The data value is: [80.11456781]
Hello from: Our Datasink ID: 68123707
Frame number: 99
The code in example3 and example4 is almost identical. This is intended!
And if you would not have put the extra effort into writing two plugins, you would have been able to make the changes from example3 to example4 in less than a minute! That is the beauty of reusing ready-made fully working components. All you need to do is put them in order, provide the data and run the pipeline!