Basic Concepts⚓︎
Actions vs Transformations⚓︎
TLDR
| Operations | Returned Type | Behaviour |
|---|---|---|
| Actions | Primitive Type or Object | Triggers the OsC on which is executed. The operation requested by the action is immediatly run. |
| Transformations | ILinqIterable |
Adds a data manipulation operation to the OsC on which it is executed. The operation requested by the trasformation will only be run if an action is requested over the operations chain |
Quote
Transformations create RDDs from each other, but when we want to work with the actual dataset, at that point action is performed. When the action is triggered after the result, new RDD is not formed like transformation.
Thus, Actions are Spark RDD operations that give non-RDD values.
The values of action are stored to drivers or to the external storage system. It brings laziness of RDD into motion. 1
With the same idea of Apache Spark in mind, there are two main kind of operations provided by the Linq Iterable Utility package:
Actions⚓︎
Actions can be seen as ending operations because they will cause the operations chain to be computed and will return the result of the computation.
The simplest Action that can come in mind is the collectToArray, which returns an array with all the items returned by the operations chain.
Other famous Actions are: some, reduce, max and min to name some of them.
The full list of Actions provided by Linq ITerable Utility can be found here.
Info
There is one exception to the Actions Definition. The materialize is an Actions even if it returns an ILinqIterable, the reason is that we need to keep alive the iterable wrapper provided by the linq-iterable-utility (since in javascript you cannot rely on extensions methods as in c#) so that is possible to continue with the data manipulation.
More in-depth about this topic can be found here.
Transformations⚓︎
On the other hand, Transformations are operations that compose the OsC and consists of a list of data manipulation
operation.
The simplest and most known Transformation is certainly the map, which is generally use to change the shape of every item of a list.
Other famous Transformations are: filter and flatMap to name some of them.
The full list of Transformations provided by Linq ITerable Utility can be found here.
Deferred Execution⚓︎
Created: March 22, 2023