Introduction
The MLOps adoption has been growing a lot in the last years, we’ve got many of news and research stand out, but other data it’s nearly 90% of machine learning projects fail in some companies themselves never deployed a machine learning model into your environment, another research shows the rise of AI into products and services where companies haven’t much time to do this adoption or jump out of the boat, neglecting your competitive market, this said, I decided to write a post to help “newbies” companies in your first step to implementing MLOps, a guide to the way into a minimal simple, security and scalable architecture.
MLOps Principles
To be a recent term and culture, MLOps has good root principles that are clear to a great part of Big techs and companies with strong maturity in MLOps, why do I say Big techs? because they are the most advanced in MLOps in the industry currently (you should see in references some articles that assert it) the firsts to explore and fail in this discipline, in fact, in most parts of this article you will see tools and practices applied by them, they influence most solutions to goes in the way.
The first principle we’ll see is The Area of MLOps, which is composed of three others tech disciplines already well-known, are Machine learning, DevOps, and Data Engineering. You can see MLOps is an intersection among other disciplines, Machine learning is a core which data to train algorithms to do some specific task, we have DevOps the part concerned to operate the model created in the productions environment which have an automation layer in the process, Data Engineering being part of collected data in several sources, data cleaning, ingestion and finally produce repository with all data cleaned and ready to build an ML model.
source: Engineering MLOps book
The second principle I’d like to list is DevOps isn’t enough. It’s important to explain the difference between DevOps and MLOps, but, aren’t they the same thing? are they not solving the same problem? the answer is NOT, DevOps came up in software development heyday to solve problem-related Software engineers and infrastructure engineers to deploy systems reliable, to solve this problem in waterfall method back then (check it out here a DevOps history article).
Which is the big difference between them? when we’re doing a machine learning model we have an additional snippet this one is DATA, DevOps was a concept to automate, avoid failure rate for new releases, and improve deployment frequency with regards to only CODE, did you catch the difference? DevOps isn’t aimed to deal with data within or external to code scripts, we have a dual way of concern in a production environment, code, and data.
In DevOps
image source: Engineering MLOps book
In MLOps
image source: Engineering MLOps book
So then, the MLOps concept was born to solve the dual way between code and data, of course, inherited some DevOps best practices (like automation, CI/CD, units tests monitoring, etc) but with an additional part which is data, in MLOps we’re focused to deliver reliable and scalable ML data-driven systems. besides other particularities that were born in MLOps which we’ll deal in next.
Said that without any further ado let’s jump into ML architecture.
Machine learning Architecture
It’s very similar to software architecture but applied to withstand ML systems, I can’t talk about ML Architecture without quoting a popular article done by the Google Engineering team MLOps: Continuous delivery and automation pipelines in machine learning which presents MLOps architecture in three different levels, inspired by this article I did this architecture based in same, with caveats in pieces of architecture, takes in count my work experience about ML Architecture as well.
source: my design architecture
I did a diagram to show the visual way to easily understand the architecture including the 5 fundamentals blocks in all ML architecture ensuring minimal reliability, scalability, and quick releases. These blocks are composed of Data Ingestion where we can gather data to be used in the model, Model Training indeed train a machine learning model to solve a business problem, Model Registry once the model has been trained we’ve to keep your artifact, Model Evaluation to ensure all metrics have been achieved in this model, Model Serving accounts for deploy a model to be used and by the end, ML Service which is just an illustration of model puts in mobile application, embedded in hardware, anything else more. Let’s jump into details.
Data Ingestion
The first layer to start is about Data Ingestion, said earlier the snippet that changes DevOps to adapt to MLOps is data, a fundamental ingredient in machine learning, real-world we’ve got a Big Data scenario when saying in Data Ingestion, to have a bunch of data available to train a model is enjoyable wherever.
Data comes to a bunch of different sources, relies mainly on your business company what kind of data you’ve got in your hands, sources like databases, CSV files, data from applications, etc. The point is how and where you can keep all this one, not only this layer but all of them is thinking on cloud or newest solutions, turns out be out of the cloud environment and try to do MLOps unreal.
In my experience in the Data ingestion layer usually, use the ETL process to deliver data in good quality to be ready to use for model training, a good tool that it’s very used in data pipelines is Apache Airflow accounts for manager steps to dirty data goes to clean data state, after that with data cleaned we will do ingestion on the data lake, enable access of this data in some repository, in the case on the data lake, we’ve plenty of tools to use as a data lake, I’ll list below some most popular of them and like mentioned earlier focused on cloud tools.
Data Lake tools: AWS S3, AWS Lake Formation, GCP Cloud Storage.
Model Training
It’s much clear what is this step of model training, all machine learning algorithms take data like input to be trained and thus solve a specific task they were trained for, in the last years we’ve seen huge advancements in machine learning algorithms, bigger architectures came up from AI research, so then, we’ve considerably strives in HPC (High-Performance Computing) to support all these new heavy model, because of it the attention point is around infrastructure to endure model training.
Nevertheless, we can train models nowadays in CPU or GPU, depending on the model architecture chosen, matter what will be your machine to run model training, all tools I’ll list below are worth it and support a range of scales that you need.
Besides that, another piece included in model training is a CI/CD concept it’s used in MLOps as well, after training a model before sending it to the model evaluation step, we need to check out some pieces of our code to endure trusty, in the list below will be tools to use for CI/CD.
It’s important to say we’re dealing with a production perspective so that we need to behave like production sense, most people that account for the train a machine learning model, create a model in jupyter notebooks, I’m mean I’m no judge to use jupyter notebooks to train a model, but in production landscape, we’ve to convert this in code script, where we can have more trusty in this one, make sure to be conversant on code scripts rather than notebooks, it’s a good practice.
Model train tools: AWS SageMaker, GCP Vertex AI, Kubeflow, Databricks.
CI/CD tools: GitHub actions, AWS CodeBuild, Jenkins, GCP Cloud Build.
Model Registry
Once the mode has been trained next step before model evaluation is to store a model artifact somewhere, it’s a well-known practice in software engineering regard to storing artifacts produces by code, in ML we need to keep a model artifact to have a tracking record about this model, for example, you’re creating a fraud detection model, fraud is banking market can be cyclic in your behave, probably this model needs be retrained in soon with another behavior in fraud data, comes up the matter to track record model versions, hyperparameters, data was used to and so on.
The model registry enables us to compare past models to new models, performance, and metrics in hand to help Data scientists to back in a loop to create a better model than before, we deliver a good and reliable way to keep info about the model in fast access.
I think (in my experience) the best tool for the model registry we’ve is MLFlow to manage all life cycles of the machine learning model, MLFlow is composed of four key components which are Tracking, Projects, Models, and Registry. Tracking has been accounted for record and query experiments, Projects are focused on code in a format to reproduce on any platform, Models are focused to deploy models in different environments, Registry is to store and manage models in a repository.
source: MLFlow
Different from other ML architecture layers I focused on say about MLFlow in the model registry layer, of course, we’ve got other solution options out there, which provides great support to your architecture solution.
Model registry tools: MLFlow, AWS SageMaker model registry, Neptune model registry.
Model Evaluation
After the model train and model registry layer we’re interested to Evaluate the model, in several projects that I’ve worked on it’s chosen a target metric in a model to improve as much as possible, some cases were Accuracy, others matter to improve F1 score, and so on, the model evaluation layer is a concern to validate if target metrics has achieved or not, the model is evaluated on a separated set of data points.
Regards tools it’s seems like a model training step, for example, you could use Sagemaker to evaluate metrics, MLFlow to compare metrics through other models, where we need to validate the metrics of the model and if isn’t good enough, we can retrain this model, following the loop again until the model performance has achieved what is expected, usually creating an evaluation environment where you receive different data to test.
Model evaluation tools: AWS SageMaker, MLFlow, Comet.
Model Serving
The last layer is model serving, like mentioned earlier most parts of ML projects failed because they don’t know about think machine learning like software, we’ve seen all parts of the components we’ve got inherent tools and mainly process that is in MLOps as well, model serving is around to deploy a model into production and available to be used by other software or an independent service.
Model serving has to ensure production testing and production releases, is needed to define a standard method to deploy, for example, you can deploy a real-time model delivering an API endpoint you need to define a standard way to put this model in API, or if deploy a batch model is needed to map your output data, things like that needed to be defined in this layer.
Some test we can cover in model serving is API endpoint latency time, how much time API is given a response to application has been consumed, in case to deploy in batch test input and output data once the batch is running verify your stats, the performance and results should be in your expect, tests are crucial before deploying machine learning models.
Model serving tools: AWS SageMaker Endpoints, Kubernetes, FastAPI, TensorFlow Serving, BentoML.
Conclusion
Once your initial ML platform is up we’ve got a follow to improve and generate the most value for the company, the base to get you there is made, there are points to improve to better ML architecture and your services deployed, it’s happening with time and maturity of MLOps team, I’d like to say this architecture was an initial provide by my experience, in the article written by Google (before mentioned) they propose a level 0 architecture which includes nothing about automation, CI/CD, infrequent releases iterations, thus MLOps has grown and nor all is the rule of thumb of, I propose this article in this way one step above to level 0.
What are the next steps? what lack of architecture? Even though we have a base to improve the platform, the next steps involve looking at CT/CM (continuous training and continuous monitoring) concepts whose was born in MLOps, retraining machine learning models are needed, usually data distribution shifts or performance model decrease and comes up the need to retrain a model, CM helps to monitor the health of the model in production.
I hope this article will be useful and clear to you all, the hope is to see more companies doing MLOps on their own, building products and services more reliable which benefit all around it, soon we’ll see great improvements regards tools and good practices, stay tuned to follow the latest releases in MLOps, see you there!.
References
- MLOps Principles: https://ml-ops.org/content/mlops-principles
- Google MLOps: https://cloud.google.com/architecture/mlops-continuous-delivery-and-automation-pipelines-in-machine-learning
- Engineering MLOps book: https://virtualmmx.ddns.net/gbooks/EngineeringMLOps.pdf
- Model registry Neptune: https://neptune.ai/blog/ml-model-registry
- MLOps tools Neptune: https://neptune.ai/blog/best-mlops-tools
65 Responses
Hello there! This article could not be written much better! Looking through this post reminds me of my previous roommate! He always kept talking about this. I will forward this article to him. Fairly certain he’ll have a very good read. Thank you for sharing!
Thanks a huge for your feedback Nicki! I’m glad to hear that, stay tuned to the next ones!
HerpaGreens is a novel dietary supplement created to help destroy the herpes simplex virus
Hi there! This post couldn’t be written any better! Reading through this post reminds me of my previous room mate! He always kept talking about this. I will forward this article to him. Pretty sure he will have a good read. Thank you for sharing!
Hey very cool web site!! Guy .. Excellent .. Superb .. I’ll bookmark your blog and take the feeds additionallyKI am happy to seek out numerous helpful info here in the submit, we want work out extra techniques in this regard, thanks for sharing. . . . . .
I truly prize your work, Great post.
Greetings! I’ve been following your website for a long time now and finally got the courage to go ahead and give you a shout out from New Caney Tx! Just wanted to tell you keep up the good job!
Puravive is a weight management supplement formulated with a blend of eight exotic nutrients and plant-based ingredients aimed at promoting fat burning.
Pretty nice post. I just stumbled upon your weblog and wished to say that I’ve really enjoyed browsing your blog posts. After all I will be subscribing to your rss feed and I hope you write again very soon!
You have brought up a very wonderful points, regards for the post.
There are some fascinating cut-off dates on this article however I don’t know if I see all of them heart to heart. There is some validity but I will take hold opinion till I look into it further. Good article , thanks and we would like extra! Added to FeedBurner as effectively
Wonderful blog! I found it while browsing on Yahoo News. Do you have any suggestions on how to get listed in Yahoo News? I’ve been trying for a while but I never seem to get there! Thank you
Please let me know if you’re looking for a writer for your blog. You have some really great articles and I believe I would be a good asset. If you ever want to take some of the load off, I’d absolutely love to write some content for your blog in exchange for a link back to mine. Please blast me an e-mail if interested. Regards!
You can certainly see your enthusiasm in the paintings you write. The world hopes for more passionate writers such as you who aren’t afraid to mention how they believe. At all times follow your heart. “He never is alone that is accompanied with noble thoughts.” by Fletcher.
I like this post, enjoyed this one thank you for posting. “When you make a world tolerable for yourself, you make a world tolerable for others.” by Anais Nin.
I?¦ll right away snatch your rss as I can not find your e-mail subscription link or newsletter service. Do you’ve any? Kindly permit me understand in order that I may subscribe. Thanks.
I love gathering utile information , this post has got me even more info! .
What is ProDentim? ProDentim is an innovative oral care supplement with a unique blend of ingredients designed to promote better oral and dental health
I’ve been browsing on-line greater than three hours as of late, but I never discovered any fascinating article like yours. It is lovely price sufficient for me. In my opinion, if all site owners and bloggers made just right content material as you did, the internet will be much more helpful than ever before.
I was very pleased to find this web-site.I wanted to thanks for your time for this wonderful read!! I definitely enjoying every little bit of it and I have you bookmarked to check out new stuff you blog post.
I would like to thank you for the efforts you’ve put in writing this blog. I am hoping the same high-grade website post from you in the upcoming as well. In fact your creative writing abilities has inspired me to get my own blog now. Really the blogging is spreading its wings rapidly. Your write up is a good example of it.
I like what you guys are up too. Such smart work and reporting! Carry on the excellent works guys I’ve incorporated you guys to my blogroll. I think it will improve the value of my website 🙂
I have read several good stuff here. Definitely price bookmarking for revisiting. I wonder how much effort you put to create this sort of fantastic informative website.
Very interesting info !Perfect just what I was searching for!
Keep working ,terrific job!
Wow that was odd. I just wrote an very long comment but after I clicked submit my comment didn’t show up. Grrrr… well I’m not writing all that over again. Regardless, just wanted to say excellent blog!
Great site. A lot of useful info here. I’m sending it to some friends ans also sharing in delicious. And naturally, thank you in your sweat!
Hey, you used to write magnificent, but the last several posts have been kinda boring?K I miss your super writings. Past few posts are just a bit out of track! come on!
Thank you for the auspicious writeup. It in fact was a amusement account it. Look advanced to more added agreeable from you! By the way, how can we communicate?
I got what you intend, regards for posting.Woh I am happy to find this website through google.
I do enjoy the way you have framed this problem and it does give us a lot of fodder for thought. However, from what I have witnessed, I really wish when the actual remarks pile on that people continue to be on issue and in no way embark upon a tirade associated with some other news du jour. Still, thank you for this fantastic point and while I can not agree with this in totality, I respect your point of view.
Hey! I know this is kinda off topic but I was wondering which blog platform are you using for this website? I’m getting tired of WordPress because I’ve had problems with hackers and I’m looking at options for another platform. I would be fantastic if you could point me in the direction of a good platform.
There are some attention-grabbing time limits in this article but I don’t know if I see all of them heart to heart. There may be some validity however I will take hold opinion till I look into it further. Good article , thanks and we wish more! Added to FeedBurner as effectively
FitSpresso: What Is It? FitSpresso is a natural weight loss aid designed for individuals dealing with stubborn weight gain. It is made using only science-backed natural ingredients.
What is ProvaDent? ProvaDent is a cutting-edge dental support supplement crafted by Adem Naturals. It integrates the BioFresh™ Clean Complex and a sophisticated oral probiotic complex to rejuvenate the oral microbiome.
I regard something really special in this web site.
Hello.This article was extremely motivating, particularly since I was browsing for thoughts on this issue last Friday.
F*ckin’ remarkable issues here. I’m very glad to peer your article. Thank you a lot and i am having a look ahead to touch you. Will you please drop me a mail?
What is Gluco6? Gluco6 is a revolutionary dietary supplement designed to help individuals manage their blood sugar levels naturally.
I love the efforts you have put in this, regards for all the great articles.
I don’t even know the way I ended up right here, but I believed this put up used to be great. I don’t know who you are but certainly you are going to a well-known blogger when you aren’t already 😉 Cheers!
Hi there, I found your website via Google while looking for a related topic, your site came up, it looks great. I have bookmarked it in my google bookmarks.
I believe that is among the so much important info for me. And i’m glad reading your article. But should observation on some general issues, The website style is great, the articles is in reality nice : D. Good process, cheers
I just could not depart your website prior to suggesting that I extremely enjoyed the standard info a person provide for your visitors? Is going to be back often to check up on new posts
This web site is really a walk-through for all of the info you wanted about this and didn’t know who to ask. Glimpse here, and you’ll definitely discover it.
I like this web blog so much, saved to bookmarks. “I don’t care what is written about me so long as it isn’t true.” by Dorothy Parker.
I haven’t checked in here for some time since I thought it was getting boring, but the last several posts are good quality so I guess I will add you back to my everyday bloglist. You deserve it my friend 🙂
Thank you for sharing excellent informations. Your web-site is so cool. I’m impressed by the details that you’ve on this site. It reveals how nicely you understand this subject. Bookmarked this web page, will come back for extra articles. You, my friend, ROCK! I found simply the information I already searched all over the place and simply could not come across. What a great website.
Dead pent articles, Really enjoyed studying.
This is a topic close to my heart cheers, where are your contact details though?
Fantastic web site. A lot of useful info here. I am sending it to a few buddies ans additionally sharing in delicious. And certainly, thanks on your sweat!
wonderful points altogether, you simply gained a emblem new reader. What would you recommend about your submit that you made a few days in the past? Any sure?
Great work! That is the type of information that should be shared around the web. Disgrace on Google for no longer positioning this put up upper! Come on over and consult with my web site . Thanks =)
Would you be involved in exchanging links?
Thank you for the sensible critique. Me & my neighbor were just preparing to do some research about this. We got a grab a book from our local library but I think I learned more from this post. I am very glad to see such fantastic information being shared freely out there.
Keep working ,impressive job!
Of course, what a fantastic blog and revealing posts, I surely will bookmark your blog.Have an awsome day!
Unquestionably believe that which you stated. Your favorite justification seemed to be on the web the simplest thing to be aware of. I say to you, I definitely get annoyed while people consider worries that they just do not know about. You managed to hit the nail upon the top and defined out the whole thing without having side-effects , people could take a signal. Will likely be back to get more. Thanks
Very nice post and straight to the point. I don’t know if this is actually the best place to ask but do you guys have any thoughts on where to employ some professional writers? Thx 🙂
What i don’t realize is if truth be told how you’re no longer actually much more smartly-liked than you may be now. You’re very intelligent. You already know thus significantly relating to this topic, made me in my view imagine it from numerous varied angles. Its like women and men don’t seem to be fascinated except it is one thing to do with Lady gaga! Your personal stuffs nice. All the time take care of it up!
I love it when people come together and share opinions, great blog, keep it up.
The core of your writing whilst sounding agreeable at first, did not sit perfectly with me after some time. Someplace within the paragraphs you managed to make me a believer but only for a very short while. I nevertheless have got a problem with your jumps in assumptions and one would do nicely to help fill in those gaps. If you actually can accomplish that, I will certainly be impressed.
I have been exploring for a bit for any high-quality articles or blog posts in this kind of area . Exploring in Yahoo I eventually stumbled upon this web site. Studying this information So i am happy to show that I have an incredibly just right uncanny feeling I came upon just what I needed. I so much definitely will make certain to do not omit this site and give it a look regularly.
Just wanna comment on few general things, The website design and style is perfect, the subject material is very excellent : D.
This web site is really a walk-through for all of the info you wanted about this and didn’t know who to ask. Glimpse here, and you’ll definitely discover it.