I'd like to train a model using Spark ML Lib but then be able to export the model in a platform-agnostic format. Essentially I want to decouple how models are created and consumed.
My reason for wanting this decoupling is so that I can deploy a model in other projects. E.g.:
- Use the model to perform predictions in a separate standalone program which doesn't depend on Spark for the evaluation.
- Use the model with existing projects such as OpenScoring and provide APIs which can make use of the model.
- Load an existing model back into Spark for high throughput prediction.
Has anyone done something like this with Spark ML Lib?