Free Trials

Download a free trial to find out which Altium software best suits your needs

How to Buy

Contact your local sales office to get started on improving your design environment

Downloads

Download the latest in PCB design and EDA software

  • PCB DESIGN SOFTWARE
  • Altium Designer

    Complete Environment for Schematic + Layout

  • CircuitStudio

    Entry Level, Professional PCB Design Tool

  • CircuitMaker

    Community Based PCB Design Tool

  • NEXUS

    Agile PCB Design For Teams

  • CLOUD PLATFORM
  • Altium 365

    Connecting PCB Design to the Manufacturing Floor

  • COMPONENT MANAGEMENT
  • Altium Concord Pro

    Complete Solution for Library Management

  • Octopart

    Extensive, Easy-to-Use Component Database

  • PRODUCT EXTENSIONS
  • PDN Analyzer

    Natural and Effortless Power Distribution Network Analysis

  • See All Extensions
  • EMBEDDED
  • TASKING

    World-Renowned Technology for Embedded Systems Development

  • TRAININGS
  • Live Courses

    Learn best practices with instructional training available worldwide

  • On-Demand Courses

    Gain comprehensive knowledge without leaving your home or office

  • ONLINE VIEWER
  • Altium 365 Viewer

    View & Share electronic designs in your browser

  • Altium Designer 20

    The most powerful, modern and easy-to-use PCB design tool for professional use

    ALTIUMLIVE

    Annual PCB Design Summit

    • Forum

      Where Altium users and enthusiasts can interact with each other

    • Blog

      Our blog about things that interest us and hopefully you too

    • Ideas

      Submit ideas and vote for new features you want in Altium tools

    • Bug Crunch

      Help make the software better by submitting bugs and voting on what's important

    • Wall

      A stream of events on AltiumLive you follow by participating in or subscribing to

    • Beta Program

      Information about participating in our Beta program and getting early access to Altium tools

    All Resources

    Explore the latest content from blog posts to social media and technical white papers gathered together for your convenience

    Downloads

    Take a look at what download options are available to best suit your needs

    How to Buy

    Contact your local sales office to get started improving your design environment

    • Documentation

      The documentation area is where you can find extensive, versioned information about our software online, for free.

    • Training & Events

      View the schedule and register for training events all around the world and online

    • Design Content

      Browse our vast library of free design content including components, templates and reference designs

    • Webinars

      Attend a live webinar online or get instant access to our on demand series of webinars

    • Support

      Get your questions answered with our variety of direct support and self-service options

    • Technical Papers

      Stay up to date with the latest technology and industry trends with our complete collection of technical white papers.

    • Video Library

      Quick and to-the-point video tutorials to get you started with Altium Designer

    Lane Recognition and Tracking with NVIDIA Jetson Nano

    Zachariah Peterson
    |  April 9, 2020
    Stealth aircraft and quantum radar

    Autonomous vehicles are slowly becoming an important part of the automotive industry. Many believe fully autonomous vehicles will soon be driving alongside humans, and technology companies are in a race to deploy fully autonomous vehicles. In December 2018, Waymo, the company that emerged from Google’s self-driving car project, officially started its commercial self-driving car service in the suburbs of Phoenix. Companies like May Mobility, Drive.ai, and Uber are following the same path.

    Autonomous vehicles might seem to be a grand vision, but semi-autonomous vehicles are already among us. New Tesla cars have the Tesla Autopilot feature, which is capable of lane recognition and tracking, adaptive cruise control, and self-parking. The ability to identify and track lanes on the road is one of many prerequisites for driverless vehicles. While lane recognition may seem like a difficult problem, you can get started with developing lane recognition and tracking algorithms with the NVIDIA Jetson Nano hardware platform.

    Intro to Jetson Nano

    The Jetson Nano COM is slightly bigger than a Raspberry Pi 3, but it can run neural networks in parallel with 472 Gflops of power. This is about 22x more powerful than the Raspberry Pi 3 and is highly power-efficient, consuming as little as 5 W. This board is perfect for embedded AI applications running on a trimmed-down Linux kernel. It has enough processing power and on-board memory for high quality image and video processing applications. Key features of Jetson Nano include:

    • GPU: 128-core NVIDIA Maxwell™ architecture-based GPU
    • CPU: Quad-core ARM® A57
    • Camera: MIPI CSI-2 DPHY lanes, 12x (Module) and 1x (Developer Kit)
    • Memory: 4 GB 64-bit LPDDR4; 25.6 gigabytes/second
    • Connectivity: Gigabit Ethernet
    • OS Support: Linux for Tegra®
    • Module Size: 70mm x 45mm
    • The developer kit is a great tool for testing a new algorithm for computer vision/image processing applications. If you’re not familiar with the Jetson Nano platform, take a look at this article. Otherwise, keep reading to see how you can easily deploy a powerful image processing application for lane recognition.
    Developer kit for NVIDIA Jetson Nano
    NVIDIA Jetson Nano developer kit. [Image source]

    Lane Recognition with Jetson Nano

    For this project, we need a Jetson Nano COM or dev board and a CSI camera (a Raspberry Pi CSI Camera v2 works fine). I’ll show this application using the Nano dev board, but you can easily build a custom baseboard for a Nano COM and deploy this application. The CSI camera will be connected to the camera port on the Jetson Nano, although you could use a USB camera or IP cameras. Below are the necessary requirements to get this project to work successfully:

    Hardware

    • NVIDIA Jetson Nano
    • CSI Camera

    Software Packages

    • Python
    • OpenCV
    • Nanocamera

    Implementation

    The full implementation for this project can be found on GitHub. Follow through the forthcoming steps to get a demo of this project up and running. These steps are important image processing steps that are intended to make lane recognition more accurate.

    Perception

    The first step of our lane detection system is being able to read live images from the camera. For new users working with the Jetson Nano, working with the cameras might be tricky. For this, we will be using the NanoCamera library. This library is not the best option for a production-grade application, but it works for a simpler application like that shown here.

    First, install the library with pip:

    .

    Reading the live image from the camera:

    .

    Remove Noise with Blur

    To make the image smoother and remove unwanted noise, we can apply Gaussian blur. This involves calculating each pixel’s value as a weighted average of the surrounding pixels.

    Gaussian blur algorithm for lane recognition
    Gaussian blur algorithm for image smoothing. [Image source]

    OpenCV includes a function for applying Gaussian blur to an image:

    .

    Color Transformation

    After the Gaussian blur, the image is still in RGB and must be transformed into HSV color space. Color separation will be used to remove unwanted colors from the image. Once the image is in HSV, we can “lift” all the unnecessary colors from the image by specifying a grayscale range.

    .

    Morphological Operations - Dilation

    Dilation adds pixels to the boundaries of objects in an image. Applying dilation to the image will help close any loose spaces in the line images.

    .

    Canny (Edge Detection)

    To detect lane lines, we can use a technique called Canny edge detection. Canny edge detection can be used to extract useful structural information from an object. The OpenCV library provides a Canny edge detection function that can be used to detect edges in an image. For Canny detection to work, we need to supply minimum and maximum thresholds. OpenCV usually recommends the value to be (100, 200) or (200, 400), so we are using (200, 400).

    .
    Canny edge detection applied to a camera image
    Image with Canny edge detection. Note the lanes shown in the image.

    Isolate Region of Interest

    The Canny edge output image contains some noise. All information other than lanes in the image can be isolated by reducing the region of interest. When performing lane recognition and navigation, we don’t necessarily need to see the entire image. An easy way to do this is just to crop out the top half of the image.

    ROI reduction for lane recognition
    Original image (left), and image with cropper region of interest (right).

    The code for reducing the region of interest is:

    .

    Detect Line Segments - Hough Lines

    After the region of interest was reduced, the lanes are clearly visible with four distinct lines. However, the computer doesn’t know that these lines represent the boundaries of two lanes. So, we need a way to extract the coordinates for these lane lines. A Hough transform is a technique used in image processing to extract features like lines, circles, and ellipses. It can be to find straight lines from a number of pixels that seem to form a line. This can be done with the HoughLinesP function in OpenCV. The function fits many lines through all white pixels and returns the most likely set of lines subject to some minimum threshold constraints.

    Lines found during lane recognition
    Line detection with a Hough transform applied.

    Here is our code that detects line segments using a Hough Transform. Most of the parameters used here can be determined through trial and error or by selecting parameters from reference images.

    .

    Combine Line Segments into Two Lane Lines

    The output of the detect_line_segments() function produces a set of small lines with endpoint coordinates (x1, y1) and (x2, y2). We need to find a way to combine them into just two lines for left and right lane lines, but how do we achieve that? One way is to classify these line segments by their slopes.

    All the left lane lines have a positive slope, and the line segments belonging to the right lane line have a negative slope. Based on these two new groups, we can take the average of the slopes and intercepts of the line segments to get the slopes and intercepts of the left and right lane lines. This is implemented with the average_slope_intercept() function:

    ..

    The make_points() function is a helper function for the average_slope_intercept() function, which takes a line’s slope and intercept, and returns the endpoints of the line segment. At this point, we now have the lane lines!

    What’s Next

    At this point, the output from this function would be input to another algorithm for controlling the speed and heading of the vehicle. In order to do this, a steering angle can be computed from the detected lane lines and then passed to a PID controller to minimize the steering angle and keep the vehicle centered.

    Although this was done with a Nano dev board, the same code provides a starting point for deploying a system in an embedded image processing system for a production environment. To do this, you’ll need to build a real baseboard for the Jetson Nano. This is easiest when you use the modular electronics design tools in the Geppetto Board Builder application. By taking a modular approach, you can easily customize a baseboard for a Jetson Nano, cameras, and other sensors for lane recognition.

    The modular electronics design tools in Geppetto give you access to a broad range of industry-standard COMs and popular modules without downloading or installing any new software. You can create production-grade hardware for a variety of applications, including lane recognition and other image processing tasks with Jetson Nano, using a drag-and-drop user interface. If your system needs additional functionality, you can include wireless connectivity, an array of sensors, and much more.

    Take a look at some Gumstix customer success stories or contact us today to learn more about our products, design tools, and services.

    About Author

    About Author

    Zachariah Peterson has an extensive technical background in academia and industry. Prior to working in the PCB industry, he taught at Portland State University. He conducted his Physics M.S. research on chemisorptive gas sensors and his Applied Physics Ph.D. research on random laser theory and stability.His background in scientific research spans topics in nanoparticle lasers, electronic and optoelectronic semiconductor devices, environmental systems, and financial analytics. His work has been published in several peer-reviewed journals and conference proceedings, and he has written hundreds of technical blogs on PCB design for a number of companies. Zachariah works with other companies in the PCB industry providing design and research services. He is a member of IEEE Photonics Society and the American Physical Society.

    most recent articles

    Back to Home