The rooster crows immediately before sunrise, the rooster causes the sun to rise.
Using GraphX to detect possible causes and effects between time related events. We observe a true causation signal by generating random correlations over same events at different time and back propagate these scores to their most connected events. Finally, we extract the most probable causes and effects together with a score of aggressiveness (how likely an event could explain downstream effects) and sensitivity (how likely an event results from an upstream cause).
Pathogen project is built for Scala 2.11.x and Spark 2.1.0.
Pathogen is available on Maven Central. Add below dependency to your pom.xml
file
<dependency>
<groupId>io.pathogen.spark</groupId>
<artifactId>pathogen-core</artifactId>
<version>x.y.z</version>
</dependency>
If you are using SBT, simply add the following to your build.sbt
file
libraryDependencies = Seq(
"io.pathogen.spark" % "pathogen-core" % "x.y.z"
)
Available as a spark package, include this package in your Spark Applications as follows
> $SPARK_HOME/bin/spark-shell --packages io.pathogen.spark:pathogen-core:x.y.z
Antoine Amend - [[email protected]]
Apache License, version 2.0