Data Access

Polling the whip for data.

Whip to the Bucket

Data Polling

Before we start incurring a whole lot of cost, we pool for the data in a repeatable fashion, using Kubernetes scheduling and Python.

The diagram below shows the Rivian API as a source, but realistically can be any, the cheat code here is if get mine for data here and grab our complex objects in this fashion, we wont eat DBU's running on the Databricks cluster to get the bronze stage in the medallion architecture.

When incorporating Kubernetes scheduling and Python into data mining workflows, it allows for efficient and mostly inexpensive deployment of data mining tasks with data sources.

for source in datasources:

GitHub - sween/rivian-databricks: Databricks Supporting Stuff for Deez WattsGitHub

Rivian API Spec

The Rivian API is a set of endpoints designed by Rivian, to enable communication and interaction between their vehicles and external software applications or systems. This API offers developers a range of functionalities, allowing access to vehicle data such as battery status, charging information, location tracking, and control over certain vehicle functions like locking/unlocking doors or pre-conditioning the cabin. Deez Watts utilizes the API to land the data passively to hopefully create innovative applications, integrations, or services that enhance the owner experience, facilitate fleet management, or provide insights into vehicle performance.

Rivian web APIs use GraphQL.

HomeRivDocs

We make heavy use of the unofficial Rivian RivDocs API Spec

Library

The Python library of choice for our data adventure is rivian-python-api which is below, it has extensive information on security and transparency to limitations. Additionally, it was primarily built off the information in the RivDocs mentioned above and works perfectly for our needs.

Unexpected error with integration github-files: Integration is not installed on this space

Below is a simple example on how to hit the Rivian API for information about your Rivian with the above library. It creates a complex json object containing, charging, status information, and owner information.

Vehicle commands scoped for write require an HMAC signature to be sent with the request. The HMAC is generated using the command name and the current timestamp, using a shared key generated from the phone’s private key and the vehicle’s public key. The vehicle’s public key is available in the vehiclePublicKey field of the getUserInfo endpoint.

import rivian_api as rivian
import os
import json

rivian = rivian.Rivian()
response = rivian.login(
    os.environ['RIVIAN_USERNAME'],
    os.environ['RIVIAN_PASSWORD']

)

# owner info, grab rivian vehicleid
owner = rivian.get_user_information()
rivianid = owner['data']['currentUser']['vehicles'][0]['id']
print(f'Rivian: {rivianid}')

# status info
whipstatus = rivian.get_vehicle_state(rivianid)

# whip info
whip = rivian.get_vehicle(rivianid)

# charging info
charge = rivian.get_live_session_data(rivianid)

# status is our main dictionary, add the other two keys
whipstatus['whip'] = whip
whipstatus['charge'] = charge

deezwatts = json.dumps(whipstatus)
print(deezwatts)