The Setup
This example is running on an Apple Mac OSX — in the ~/.bash_profile, add the following lines:
export SPARK_HOME=/Users/…/…/spark-2.3.0-bin-hadoop2.7/
export SPARK_SUBMIT_OPTIONS=”–packages com.coherentlogic.cmr.api:cmr-api-core:2.0.2.1-RELEASE,dom4j:dom4j:1.6.1,com.fasterxml.jackson.core:jackson-databind:2.9.6 –exclude-packages junit:junit,org.jboss.spec.javax.ws.rs:jboss-jaxrs-api_2.0_spec,org.jboss.spec.javax.servlet:jboss-servlet-api_3.1_spec,org.jboss.spec.javax.annotation:jboss-annotations-api_1.2_spec,javax.activation:activation,net.jcip:jcip-annotations,org.jboss.logging:jboss-logging-annotations,org.jboss.logging:jboss-logging-processor”
For the 2.0.3-RELEASE we use the following:
./spark-shell –packages “com.coherentlogic.cmr.api:cmr-api-core:2.0.3-RELEASE,dom4j:dom4j:1.6.1,com.fasterxml.jackson.core:jackson-databind:2.exclude-packages “junit:junit,org.jboss.spec.javax.ws.rs:jboss-jaxrs-api_2.1_spec,org.jboss.spec.javax.servlet:jboss-servlet-api_3.1_spec,org.jboss.spec.javax.annotation:jboss-annotations-api_1.2_spec,org.jboss.logging:jboss-logging-annotations,org.jboss.logging:jboss-logging-processor,org.jboss.spec.javax.xml.bind:jboss-jaxb-api_2.3_spec,org.reactivestreams:reactive-streams,javax.activation:activation,net.jcip:jcip-annotations,javax.validation:validation-api,javax.json.bind:javax.json.bind-api,javax.ws.rs:javax.ws.rs-api”
The Example
import com.coherentlogic.cmr.api.builders.CMR
import com.coherentlogic.cmr.fred._
val FRED_API_KEY = System.getenv(“FRED_API_KEY”) // You can acquire your own FRED API key for free directly from the Federal Reserve Bank of St. Louis.
val cmr = new CMR ()
val sp500CaseShillerUSNationalHomePriceIndexDS = cmr.fred.series.observations.withApiKey(FRED_API_KEY).withSeriesId(“CSUSHPINSA”).doGetAsObservationsDataset(spark)
sp500CaseShillerUSNationalHomePriceIndexDS.show
The Result
