Author: Aaron Alvarez

Posted on

Anomaly detection with accelerometer data

In last week’s blog post, we covered getting our accelerometer data and verifying it. In this blog post we will wrap up this project, by collecting the rest of our dataset and then training and testing our model.

Project goals:

This project is one that we have been working on consistently over the past few months. The main goal is to create a machine learning algorithm that can learn to identify anomalies in sets of data that the human eye would otherwise not see. Furthermore, we want to make it easily applicable to different sets of data. As long as you have “normal” data, you should be able to implement the algorithm.

Equipment used:

  • Jetson Nano
  • BMA220 Accelerometer
  • AC motor (or any motor with significant vibration)
  • JupyterLab

Collecting anomaly data:

Last week we covered how to get x, y, and z output. Many industrial plants where this kind of predictive maintenance is needed are always collecting their data. They have logs of normal and failure logs, which we will be emulating. The data we collected from the normal running motor will be used as our “normal” data. 

However, we still need our “anomaly” data. To artificially inject an anomaly, I recorded about 100 data points from the accelerometer where I was shaking it. This data looks subtly different from our “normal” data. I then marked each data point as either 0 (normal) or 1 (anomaly), and shuffled the whole dataset. This is what it should look like:

We now have a full dataset of about 1000 “normal” labeled data points and 100 “anomaly” labeled data points. 

Model creation and training:

Firstly, we need to import some libraries. Use these import statements to import libraries we need for data handling, obtaining accuracy, and just creating our model. 

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from sklearn import datasets
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
from sklearn.ensemble import RandomForestClassifier

We can then read in our .csv file that contains our data, using this line. Note that your .csv might be named differently. 

df = pd.read_csv('vibrationdata.csv')

We can use df.head() to make sure our data is properly imported. You should see an output like this:

Next, we need to split our test and training data, as well as split our features and labels. Use these lines to split our features and labels into an x and y array, and then into a training set with 70% of the data and a testing set with 30%.

x=df[['x','y','z']]
y=df['status']

x_train, x_test, y_train, y_test = train_test_split(x, y, test_size = 0.3)

We can use the following code to create an instance of a Random Forest Classifier model, which was deemed the most accurate in our past testing. This instance uses 100 decision trees, as declared by the n_estimators parameter.

model = RandomForestClassifier(n_estimators=100)

Finally, we can fit our model with our training x and y datasets. 

model.fit(x_train, y_train)

Testing the model’s accuracy:

We can use the following line to get our prediction for our testing data. 

y_pred = model.predict(x_test)

Then use the following to compare the accuracy between our prediction and the actual output. 

print("Accuracy:",accuracy_score(y_test, y_pred))

For my dataset, our model yielded an accuracy result of 95%. 

Future usefulness of the model:

This model setup is very useful for the simple fact that it can be used with many different types of data, as long as you can obtain some “normal” data and a few anomalies. 

With the implementation of live data processing, you will have a model that can alert you as soon as anything different occurs. And as long as you keep updating the model with new training data, it will be able to detect new problems before they become costly. 

This concludes the main goal of our project, although there is still room for improvement. With more extensive data collection, we could get an even more accurate model. 

Posted on

Getting input for predictive maintenance 2

In last week’s blog post, we went over our plan for collecting data for predictive maintenance. We also looked at the hardware configuration and began setting that up. This week, we will finalize getting our output data and do some final steps before we can train our model.

Handling x, y, and z output:

In an older blog post, we created a python program that will read the x, y, and z data from our BMA220 accelerometer and print it out. To use this data in our machine learning model, we would need it to be put into a .csv format. We will need to update our python code to do this. 

Our main function should look something like this: 

while (True):
        xdata = i2cbus.read_byte_data(
            i2caddress, 0x4)  # read the value of x data
        ydata = i2cbus.read_byte_data(
            i2caddress, 0x6)  #read the value of y data
        zdata = i2cbus.read_byte_data(
            i2caddress, 0x8)  #read the value of z data
        data = [xdata, ydata, zdata]
        print(data)  # print the value of x y and z data

We can add this code inside our while loop to create a .csv file named outputdata, and write each loop iteration’s data to the outputdata csv:

with open('outputdata.csv', 'a') as file:
            writer = csv.writer(file)
            writer.writerow(data)

To keep track of how many rows of data are added, we can create a variable called datacount outside our while loop:

    datalog = 0

We can add the following code at the end of our while loop. This increases the datacount variable by one each time the loop adds an entry to the outputdata csv. It then prints out the number of entries that have been recorded in the outputdata csv:

datalog += 1
        time.sleep(1)
        print(datalog, "entries recorded")

After implementing all these changes to our python code and running it, a file called outputdata.csv should be created. Depending on how long you let the program run, the file should look like this with more or less entries.

8,12,56
8,12,56
12,8,60
8,8,60
8,8,60
4,8,56
12,8,60
4,8,56
8,12,56
8,12,60
8,8,60
8,8,60

The values may differ depending on how your accelerometer is positioned. 

Analyzing our data:

When I first looked at the x, y, z data in the outputdata file, I was surprised by the large changes I saw, when the accelerometer was seemingly not moving. I wasn’t sure if this data was normal and just due to noise, or if it was a defective accelerometer. To get a better understanding of it, I plotted the data to see if it was moving within a certain range, or if it was just all over the place.

Doing this is good practice in any case to make sure our training data isn’t full of strange outliers.

  • Plotting the data:

To plot our x, y, and z data, we will use the matplotlib library. Specifically mplot3d and pyplot from it. Include this code at the top to use it:

from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt
import pandas

Then, we will use pandas to read our csv file and put it into a pandas dataframe:

points = pandas.read_csv('outputdata.csv')

Use the following piece of code to create a figure and separate our data into x, y, and z.

Note that to do this you will need to add x,y,z as the first entry in the outputdata csv.

fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')

x = points['x'].values
y = points['y'].values
z = points['z'].values

Finally, we can plot it as a 3D scatterplot. 

ax.scatter(x, y, z, c='r', marker='o')

plt.show()

We should get an output like this: 

This shows us that our data mainly is the same points over and over, which shows that it’s mainly due to noise and not a faulty accelerometer.

In the next blog post, we will make sure we properly collect test and training data, and begin to feed it to our model. 

Posted on

Getting input for predictive maintenance 1

In last week’s blog post, we worked on setting up our accelerometer to input data to our jetson nano. We can now access the x, y, and z data of the BMA220 accelerometer. This week, we will cover our plan to implement the BMA220 for use with predictive maintenance.

The configuration:

The main components in this project are the BMA220 accelerometer, the jetson nano, an arduino, and an ac/dc motor. They will be configured as shown below.

The jetson nano will process the input from the BMA220, which will be fixed on to an ac/dc motor. The motor’s speed will be controlled by the arduino. In this project, it is important that all the components are fixed to a board. Any slight play in their stationary positions may throw off the data collection.

Collecting data:

To do any kind of machine learning, we first need a large set of data. In this case, when we are trying to predict failure, we will need ‘normal’ data, and ‘failure’ data. This is data taken when the machine (in this case a motor) is running fine, and when it is in a failed state or abnormal state. 

We will run the motor 24/7 for a week or so to collect a lot of normal data. Then, we will inject problems that cause the data to become abnormal. This can include things like messing with the bearing, bumps and shakes, etc. We will separate the datasets, train the model to recognize any abnormal data, and to notify us before something costly could happen.

Why is this useful:

Having a model like this is useful for many reasons. The main one is interchangeability. A model like the one we are making, that is trained to spot any abnormal data, can be implemented on various pieces of machinery. All that is needed is a large set of data, and since most machinery records its data anyway, this is easy to get. 

While you could theoretically hard-code all the rules to recognize failure for a certain machine, this is a lengthy process that will require a lot of human eyes looking over huge datasets. It quite literally is like searching for a needle in a haystack to find the triggers that signal the machine’s failure. With machine learning, you can easily be notified of any common failures, and even new failures that have never occurred before.

In next week’s blog post, we will have our motor and will be able to create our project for data collection. 

Other useful resources:

  • Machine learning techniques

There are various ways to implement machine learning in predictive maintenance. The one we are using here is to flag anomalous behavior. This article goes over the other ways ML can be used and how the steps differ.

Posted on

Getting data from BMA220 with Jetson Nano

In last week’s blog post, we worked on connecting an accelerometer (BMA220) to the jetson nano thru I2C. This is just one of our steps towards the goal to eventually create a predictive maintenance model that can take an input of various different types. In this post we will continue on this path by getting the x, y, and z data from our accelerometer with python.

Using SMBus:

To communicate with our sensor, we will be using SMBus, or System Management Bus. SMBus is a two-wire bus made for working with I2C. The documentation can be found here. To use SMBus in our python file, first pip install smbus, then add the following code to the top:

from smbus import SMBus
import time #this allows us to use time-related functions we will need later.
  • Getting the BMA220’s address

To communicate with the accelerometer, we will need to specify the address of it, and what bus it is on. We can use the i2cdetect command below to scan a specific bus to find the accelerometer’s address. 

i2cdetect -y -r 1 #scans bus number 1

You should get an output similar to this:

This tells us that the accelerometer is located on bus 1, at the address 0x0a. We can take these pieces of data and create two variables in a python file:

i2cbus = SMBus(1)  # Create a new I2C bus
i2caddress = 0x0a  # Address of BMA220
  • Reading data output

We will next use SMBus to read the x, y, and z data from the corresponding register address. Below is a global memory map that shows the BMA220’s I2C register addresses and their functions. The full documentation of the BMA220 can be found here.

The useful addresses for us here are 0x4, 0x6, and 0x8. These correspond to the x, y, and z data, respectively. 

To communicate with these addresses we will use the “read_byte_data” function (which takes the parameters ‘i2c address’ and ‘register to read’) inside a loop that runs forever. We will get the x, y, and z data and assign them to a variable. We will print out the three values once every second. This is the code to implement this: 

while (True):
        xdata = i2cbus.read_byte_data(
            i2caddress, 0x4)  # read the value of x data
        ydata = i2cbus.read_byte_data(
            i2caddress, 0x6)  #read the value of y data
        zdata = i2cbus.read_byte_data(
            i2caddress, 0x8)  #read the value of z data
        print(xdata, ydata, zdata)  # print the value of x y and z data
        time.sleep(1)

After running the python file, you should get an output similar to this:

If you physically move the accelerometer, you can see the values change as they are updated every second. 

Potential problems:

  • Strange addresses on buses

If you run i2cdetect on bus 0, or bus 2, you may see addresses there although you don’t have any other I2C devices connected to said bus. These are actually internal I2C devices that the jetson nano uses for internal communication. This was confusing for me when locating my i2c device, as I wasn’t sure if it was on the bus I was thinking of. A full discussion of the problem can be found here.

Other useful resources:

  • Eclipse UPM sensor repository

There exists a repository created by the Eclipse foundation which is supposed to provide drivers for a variety of sensors, including the BMA220. Personally, I could not figure out how to use it, but there is valuable information in the /src/bma220 folder pertaining to the BMA220’s registers, specifically in the .hpp file. The project can be found here.

  • I2C guide

This blog provides a good overview of I2C, and a brief tutorial for using an I2C device with a Raspberry Pi. The code given can be used on the jetson nano with only a few small tweaks.

  • Nvidia Developer Forums

If you have a unique problem using I2C with the jetson nano, chances are there is someone else who has had the same problem and posted about it on the Nvidia developer forums, found here.

Next week, we will take a look at setting up an experiment to use the accelerometer(s) to get useful data. 

Posted on

I2C Input on Jetson Nano

In last week’s blog post, we finished creating our first machine learning model using a public repository dataset. While this is a major step towards our end goal, it is missing a critical aspect: being able to use our own data on the model to train it and test it. The goal for the model is to eventually be able to apply it to many different types of data, and to easily be able to change in between use cases. 

Collecting our own data:

To use our own data on the model, we first need to collect some data. In this case we will be using an accelerometer that outputs x, y, and z data. We are using the SEN0168 accelerometer with a BMA220 chip. This accelerometer uses an I2C or inter-integrated circuit interface to connect to the jetson nano.

  • What is I2C?

I2C is actually not eye-two-C, but I-squared-C. This stands for “inter-integrated circuit interface”. I2C is a single-ended serial communication bus that is widely used for connecting integrated circuits (which can come on all kinds of sensors, motors, etc.) to processors and microcontrollers. Regardless of how many pins you have on your microcontroller, you can connect as many I2C devices as you’d like. I2C even supports connecting multiple microcontrollers to allow for more than one controller to communicate with all peripheral devices on the bus. 

  • Connecting to the Jetson

The jetson nano comes with a 40-pin header. The pins functions are shown in the image below.

For the I2C connection, we will only need to use 4 pins: VDC (power- in this case 3.3v), GND (ground), SCL (clock), and SDA (data). These pins on the jetson nano (pins 1, 3, 5, and 6) will be plugged into the corresponding pins on the BMA220.

Checking the connection:

If the BMA220 has been properly connected, it should light up. We can now check for the signal on the jetson nano. 

Open the command line and run “ i2cdetect -y -r busnumber”. I am connected to I2C bus 1, and if you followed the same pin setup described above you should be as well. 

We can see that we are detecting our I2C device on 0x0a, or port 10. If you connect to other buses, you can check that as well by changing the number of bus in the i2cdetect command. 

We now have a properly connected I2C device to our Jetson Nano. In the next blog post, we will cover any configuration necessary and write some code to actually access the accelerometer’s data. 

Posted on

The Roadmap to Model Development Part 3

In last week’s blog post, we finished up the third step (Data Preprocessing for Prediction) and began creating our first models in the final step (Model Development). Today, we will create four more models, and test each of them to compare their accuracy and runtimes.

Creating the models

Last blog post, we created only the Logistic Regression model and the K-Nearest Neighbors model. We will add the following four models today: Support Vector Machine, Decision Trees, Naive Bayes, and Random Forest. We will use the Sci-kit learn library for all of these.

  • Support Vector Machine

In this step, we create the Support Vector Machine model and name it svc. We then append its full name to our ‘classifier’ array, and the instance to our ‘imported_as’ array. The Support Vector Machine model works by separating the data points into ‘hyperplanes’ that have optimal decision boundaries.

  • Decision Tree

Here is the code implemented to create the Decision Tree model. It also appends the name and instance to the ‘classifier’ and ‘imported_as’ arrays, respectively. The Decision Tree model works by creating a ‘tree’ with multiple branches, with a binary question, leading to more branches.

  • Naive Bayes

In this code block, we create the Naive Bayes model and append its name/instance to the ‘classifier’ and ‘imported_as’ arrays. Naive Bayes is a model that attempts to perform classification by assuming all features are independent of each other. 

  • Random Forest Classifier

This is the code to create the Random Forest Classifier Model. We then append it to the ‘classifier’ array and the ‘imported_as’ array. The Random Forest Classifier works by creating multiple decision trees and weighing all their results before a decision.

Testing and measuring accuracy

We can now create our main class, that we will use to create an object with all the models so we can easily compare their accuracies. 

We create a class called ‘Modeling’ in which we have a few functions, ‘fit’ and ‘results’ these two can be called on the ‘Modeling’ class object to fit the models, and to get the results, respectively. Now, we will make an array that holds all the models we want to test. We could also use the ‘imported_as’ array, but for simplicity we will make a new array and name it ‘models_to_test’.

Finally, we can create an object of the ‘Modeling’ class and use our ‘fit’ and ‘results’ functions on it. We will use our previously made arrays of x and y training data, x and y testing data, and our ‘models_to_test’ array as the arguments. 

If it runs properly, you should be returned with ‘name of model’ has been fit for each of the models, and a pandas dataframe that holds the accuracy and runtime of each model. 

The Random Forest Classifier seems to be the most accurate. If you want to test this multiple times to see if the RFC model is ever beat, you can use a simple for loop to get the top model as many times as you’d like.

You can run it as many times as you like, and you’ll see that the Random Forest Classifier is always the most accurate. However, this comes with a price of a significantly higher runtime than the other models (0.4 seconds vs 0.01 and lower). This is why we trained multiple models, so we could compare which one would deliver enough accuracy while still being appropriate for the hardware it will run on. From these models, you can choose which one is most appropriate for your use case.

What’s next?

We now have a few accurate models to choose from for the purpose of predictive maintenance. However, the data used to train and test them is not our own. This project is meant to be easily adaptable to different sets of data, so next we will collect our own data for predictive maintenance and attempt to employ the model(s) on that dataset.

Posted on

The Roadmap to Model Development Continued

Last week we took a look at the first two major steps in the process of model development, Data Preprocessing and Exploratory Data Analysis. In today’s article we will continue on the same roadmap, looking at two more steps that are arguably more interesting. These include the final data preprocessing to prep for model development, and developing the first few models.

Data Preprocessing for Prediction

What is the difference between the first data preprocessing step and the second? In the first step, you will remember that all we did was drop unwanted features. This is solely to prepare the data so we can perform Exploratory Data Analysis. In the second preprocessing step, we do all the other necessary actions for model deployment. 

  1. Encoding categorical features

Label encoding refers to converting the labels into a numeric form so as to convert them into the machine-readable form. We will use it on our training data, and using the “.fit” sci-kit method it will figure out the unique values and assign a value to it, returning the encoded labels.

This is what the code looks like:

  1. Splitting Test and Training Data

Every machine learning model needs test and training data to learn and improve. In this step, we separate the test and training data from the dataset consisting of ten thousand values. We use the “train_test_split” module from sci-kit and “.drop” from the pandas library to get four datasets, x_train, y_train, x_test, and y_test.

This is what the code looks like:

  1. Feature Scaling

Feature Scaling is a technique to normalize/standardize the independent features present in the data in a fixed range. It is done to handle highly varying magnitudes or values/units. Without feature scaling, a machine learning algorithm will weigh greater values higher and almost disregard smaller values, not taking into account the units. Note that we only do this to the x datasets, because the output does not need to be normalized.

This is what the code looks like:

Here is an image that shows how the distribution of the data is changed after feature scaling. 

Model Development

One of the final and most important in any model’s development, the actual creation of the model itself. For the best accuracy possible, we will be creating multiple models and comparing their accuracy to choose the best one. In this week’s post we will look at a couple and next week we will look at the rest. We will first create a place to store our model’s names and their actual instances.

Here is the code I used to do this:

  • The Logistic Regression Model

The Logistic Regression model is a supervised learning algorithm that investigates the relationship between a dependent and independent variable and produces results in a binary format (fail or not fail). This is used to predict failure.

Here is the code for the linear regression model, and for appending it to our arrays.

  • K- Nearest Neighbors Model

K-Nearest Neighbors is also a supervised machine learning algorithm which is mostly used to solve classification problems. It stores all the available cases and classifies new cases based on similarity. “K” is the number of nearest neighbors.

This image shows how k-nearest neighbors uses the location of data distribution to predict.

This is the code to implement the KNN model and append it:

In the next week’s blog post, we will finish the final step by creating more models, including Support Vector Machine, Random Forest, Naive Bayes, and Decision Trees. We will then test the accuracy of all of them to choose the most accurate one.

Posted on

The Roadmap to Model Development

Last week, we looked at how Machine Learning can be useful in industry. Today we will explore the path to machine learning model development and the individual steps that go into that. The above roadmap outlines the four major steps in yellow on the left, and the smaller substeps on the right.

What is Data Preprocessing?

Data Preprocessing is one of the most important steps in tackling a machine learning problem. Depending on your dataset, you may have problems like missing values, useless features, and other types of noise. In the data preprocessing step, you focus on removing features that hold useless data, and addressing the missing values. We will use the Pandas library to work with our data. This is what our raw, unprocessed data looks like.

We can see that there are some features that would provide nothing of value to the machine learning model. These are features like the UDI and Product ID. They are just labels used to name each entry, so they are not actually useful. After removing them, our data is much more model-ready.

In our specific dataset, there were no missing or null values. In the real world this is usually not the case, and you should remove the features(columns) with null values entirely.

What is Exploratory Data Analysis?

Exploratory Data Analysis is a key step that involves the initial investigating of your dataset to find anything unusual or helpful. For example, when performing EDA, you may see that Feature X correlates directly with the output data. This can be helpful when selecting a model, and the inputs that go into the model. Exploratory Data Analysis involves a lot of charting and graphing to gain a better understanding of the summary of your dataset.

Using the .describe()  function from the Pandas library is useful to quickly view the stats of your data. You can see the mean for each numerical feature, the standard deviation, min and max, and percentiles.

Another important part of EDA is analyzing the skewness of your data. Using .skew() from the Pandas library, we can see just how skewed each feature is. If the skewness is between -0.5, and 0.5, the data is almost symmetrical. If it’s between -1 and 0.5 (negatively skewed) or between 0.5 and 1 (positively skewed), the data is slightly skewed. And lower than -1 or greater than 1 the data is extremely skewed.

Using Plotly, you can make many different graphs to view your data.

You can mix and match features to see if there is correlation. Here we compare air temperature and failure type.

Keep in mind that Exploratory Data Analysis is meant just that, Exploratory. There is no need to compare every single feature, it is only your initial exploration. EDA is not only useful to you, but it’s good practice so other engineers can look at your notebooks and easily understand it as they read along.

In the next blog post we will return for the third step in our Roadmap To Model Development which includes more data processing. Then, we will finally train and test our model(s).

Posted on

Machine Learning in Predictive Maintenance

Last week, we looked at the deployment of applications on a Kubernetes cluster. This was one step in preparation of a bigger project, one which involves using machine learning(ML) for predictive maintenance of industrial processes. 

What is Predictive Maintenance?

Predictive maintenance is the process of using historical data to find patterns that can help to predict future failures of industrial processes, AKA machine failure. Traditionally, predictive maintenance has been done by hard-coded rules that say things like, “when x rotations exceeds y threshold, this machine will fail.” The problem with these hard-coded rules is that they all have to be created by patterns that are readily apparent from the data (since they are written by humans).

How Machine Learning can help

Machine learning algorithms can pick up on patterns that would not be visible to the human eye. A ML model can take in a lot of data at a time and use it to “train” itself to find patterns from data that is otherwise unrelated. 

Without ML facilities are prone to performing unnecessary maintenance. By implementing machine learning, you can not only know when things are going to fail, but you can also reduce the wasted money that comes from over-maintenance. This will reduce cost while not risking downtime or the safety of workers. 

There are multiple ways that ML models can be implemented for predictive maintenance purposes. We will explore two common types.

Regression Approach:

This approach is used to predict the Remaining Useful Life (RUL) of an asset. It will tell how many days or cycles are left before a system fails. To implement this, you need static and historical data, with every event labeled. You can only implement this if there is only one type of failure. If there are multiple, you will need multiple models. This is the well-known linear regression approach, which finds a line of best fit according to the data provided.

Classification Approach:

This approach is used to predict if a machine will fail in the next N days or cycles. This can be more useful and accurate than using a regression, because you may only need to know if the system will fail soon. Implementation also requires static and historical labeled data. 

What are we trying to do?

Our approach involves time-series analysis to forecast when machines are due for maintenance.  Alongside that, a Remaining Useful Life model will most likely also be implemented just to have more information. For now, we are using free datasets available at the UCI Machine Learning Repository (UCI Machine Learning Repository) that are specifically made for the purpose of predictive maintenance. Once we have a working model, we can collect our own data and use that instead.

The model will be deployed on the Kubernetes cluster for purposes of redundancy.

Next week we will further explore the creation of our machine learning model. We will discuss cleaning and preparing data, and designing neural networks.

Posted on

Roadblocks to the first Deployment (K3s)

Last week, we looked at setting up a Kubernetes cluster on three Jetson Nanos, to prepare them for application deployment. That’s what we tackled this week, and in today’s blog post we will look at the obstacles we’ve encountered.

For these examples, we have been trying to deploy a test application using the instructions from the Rancher docs. This uses a pre-built container provided by Rancher. To recreate my deployment do the following.

  1. Create a file called testdeploy.yaml and paste the following inside:
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mysite
  labels:
    app: mysite
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mysite
  template:
    metadata:
      labels:
        app: mysite
    spec:
      containers:
        - name: mysite
          image: kellygriffin/hello:v1
          ports:
            - containerPort: 80
  1. Once you’ve saved the file, use the following command to run it. 
kubectl apply -f testdeploy.yaml
  1. View your deployment using:
kubectl get pods

The CrashLoopBackOff error:

If you are running the above steps on a Jetson Nano cluster as well, you may see the common CrashLoopBackOff error.

NAME                      READY   STATUS             RESTARTS      AGE
mysite-57b5b46f97-rfcgx   0/1     CrashLoopBackOff   5 (39s ago)   3m30s

This error is well documented for Kubernetes in general, and is often caused by insufficient resources, trying to access a locked file, or a locked database. However, in the case of the Jetson Nano cluster, I believe the reason for the CrashLoopBackOff error is the ARM_64 architecture it runs on and shares with the new M1 Macbook Pro. A lot of pre-made containers may be designed specifically to be run on x64 architectures or even x86 making them incompatible with the ARM_64 architecture. 

To test this theory, I tried to deploy the Nginx service on the cluster. Nginx should work as it is made to work with ARM_64 architectures. I ran the following, to create a deployment of Nginx using the Nginx image. 

kubectl create deployment nginx --image=nginx

After running kubectl get pods, we can see that the Nginx service is running fine on the cluster, while the testdeploy continues to restart over and over after crashing.

NAME                      READY   STATUS             RESTARTS        AGE
mysite-57b5b46f97-rfcgx   0/1     CrashLoopBackOff   7 (2m31s ago)   13m
nginx-85b98978db-gc6ms    1/1     Running            0               20s

Many premade containers available to use for testing may be designed for different architectures from the ARM_64, which caused the frustrating CrashLoopBackOff error. As long as your Nginx deployment works, your cluster should be working, it will just require learning and much trial and error to deploy properly. 

The ImagePullBackOff error:

The ImagePullBackOff error is a very finicky error. I encountered it the first few times trying to deploy the Nginx pod using the steps outlined above. 

NAME                      READY   STATUS             RESTARTS      AGE
mysite-57b5b46f97-rfcgx   0/1     ImagePullBackOff   0 (9s ago)   4m30s

At first I had no idea how to get around it, and I assumed it was a problem with the place it was pulling the image from. Maybe the image was locked, or corrupted I thought. But after retrying multiple times, I let it run for 8 or so minutes. It retried pulling multiple times, and then one time it just worked. If you’re encountering this error, I recommend letting it run for at least 10 or so minutes before trying something else. 

Those are a few of the more problematic errors I encountered when trying to get to the first deployment. The next step will be to deploy a machine learning model on the cluster. This will require a lot more in terms of containerization, and we will explore that containerization and the creation of the model itself next week.