Ag Tech Sunday - Robotics will shape Agricultures future

This article is bought to you by Entegra Sheds.

Technology advances ranging from better autonomous vehicles to smarter machines will help relieve the effects of an aging agricultural workforce and a shrinking supply of workers.

By Tanya M. Anandan

Intuitive touchscreen displays put a wealth of real-time data and analysis tools at operators’ fingertips so they can monitor, adjust and learn from their machinery while moving through the field. Courtesy: John Deere/RIA

The issue of agriculture sustainability is a people problem. However, it might be the robots that save humanity. Automation and artificial intelligence (AI) will help relieve the effects of an aging agricultural workforce and a shrinking supply of field workers looking for less strenuous work. Self-driving agricultural machinery and autonomous drones mean farmers can spend less time watching the path in front of them and more time focusing on the path ahead to more sustainable harvests and profits. Data mining and predictive analytics will become common tools of the trade, enabling farmers to make better decisions, maximize resources and optimize yields.

Robots and machine learning are helping facilitate new, more sustainable agricultural methods that take farming inside and to new heights to conserve resources, minimize chemicals and shorten time to market. With more sustainable, fresher options from traditional growers, greenhouses and vertical farmers, the world’s population should be able to eat better, cleaner, smarter and more affordably.

The farm of the future is high-tech, better informed, and empowered to produce more with less resources for a more sustainable future for all of us. A good place to start is the granddaddy of farming technology, which has stood the test of time.

From plow to precision agriculture

In 1837, Illinois blacksmith John Deere crafted the first commercially successful cast-steel plow from a broken sawblade. Highly polished steel and a contour design made the plow an ideal tool for the thick, clay-laden soil of the Midwest’s virgin prairies. Deere’s invention would set the stage for his namesake company.

After 180 years, John Deere is a global manufacturer of machinery for the agricultural, construction and forestry industries with annual sales of $37.4 billion. The Moline-based Fortune 500 is world renowned for its ongoing commitment to providing innovative products and services to support those linked to the land. In 2017, Deere released the S700 Series Combines, the latest in smart grain harvesting technology.

Self-steering combine uses sensor fusion, robotics and artificial intelligence to automate many harvesting tasks for precision farming. Courtesy: John Deere/RIA

“It actually uses imagery to identify when an individual kernel of corn is being damaged,” said Joel Hergenreter, automation strategy lead for Deere’s Precision Farming Group. “The robotics knows how to adjust the combine to ensure that individual kernels are not damaged going forward.”

The S-Series made a big splash on the Las Vegas Strip at the Consumer Electronics Show this past January. Sporting Deere’s green-and-yellow color combo, the gigantic multi-ton combine stopped CES showgoers in their tracks. But what’s a “tractor” doing at a consumer electronics trade show? This, however, is no ordinary machine. The combine has onboard robotics and AI, and is self-steering.

But the real showstopper? Autonomous driving is not new to Deere. The Ag giant has been in the business of self-driving technology for two decades.

In 1999, Deere acquired NavCom Technology, an early innovator in advanced GPS technologies. Four years later, Deere introduced the AutoTrac guidance system, which uses GPS with real-time kinematics (RTK) corrections to precisely guide a large agriculture machine through a field.

Since AutoTrac’s introduction in 2003, Deere has added mechanical sensors and vision sensors (cameras) to identify crop rows and ensure the sprayers and harvesters drive between the rows and do not damage crop. Sensor fusion is used to combine the signals from the GPS receiver, mechanical sensors and vision sensors to automate steering.

Today, most of the company’s large agriculture equipment is “self-driving capable.”

Ad - Make you next Shed an Entegra Shed - The best in the business - Ad 

“We’re down to sub-inch accuracy in being able to drive a 20-ton machine through the field,” Hergenreter said. “It’s the GPS receiver that enables us to get to sub-inch accuracy. We’re in over a hundred countries with that product. Our farmers are now able to farm 14 to 20 hours a day in their equipment during critical times of the year when the weather windows are very tight for a given operation. Previously they would be fatigued from the long days, which would cause them to have to stop planting or spraying during these tight windows. AutoTrac allows them to start earlier in the day and go much longer into the evening.”

Even though a human operator is still behind the wheel, automation allows for more accuracy among the narrow crop rows. When humans get tired, repeatability and accuracy suffers.

“With additional sensors, we’re allowing farmers to increase their speed of operation by up to 50%,” Hergenreter said. “Instead of driving 8 miles an hour, some are now able to drive 10 or 12 miles an hour through the field, which really improves their productivity throughout the day.”

However, an operator is still required in the cab; there are many more tasks yet to be automated. First it was steering. Now, with more sensor inputs, robotics and AI, Deere is moving toward making sure the crop is healthy, plant by plant.

AI-enabled weed control

Precision weed control gets a boost from AI. Using robotics and machine learning, farmers can pinpoint the application of fertilizers and herbicides.

In 2017, Deere expanded their agricultural arsenal with the acquisition of Blue River Technology, developer of the lettuce bot, an automated weed sprayer and the forerunner to their latest system. It uses computer vision with machine learning and advanced robotics to distinguish between a crop and a weed, and only spray the weed.

From inside the machine’s cab, operators have a clear 360-degree view of the field, easy access to multifunction controls and programmable shortcut buttons, and options for real-time adjustments to increase harvesting efficiency and ease of use. Courtesy: John Deere/RIA

“The machine processes images at a rate of one image every 50 milliseconds,” Hergenreter said. “It compares those real-time images to a library of over 300,000 images, making sure only the weeds are targeted.”

This dramatically reduces the amount of herbicides used. Field tests have reported using only 10% of the herbicide needed in the past. The concept can be reversed to precisely apply fertilizer to only desired plants, thereby reducing waste while optimizing yields.

Big Data for better decisions and a better crop

Data is one of the most valuable assets for farmers. Precision agriculture feeds on big data. Today’s farmers can use web-based tools to help them create prescriptions, or maps, of how much fertilizer to apply to certain areas of the field. That prescription can then be sent to the sprayer, and using GPS as it drives through the field, the sprayer will automatically adjust the rate to ensure the right amount of fertilizer is applied to a specific area.

All of this data exchange requires a lot of computing power. Deere has not only had to transition from traditional agriculture to precision farming with advanced robotics and AI, they have also transformed their knowledge base and resources to support Internet of Things (IoT) solutions, mobile apps and cloud services.

Hergenreter said computation takes place on two levels. One is on the Ag machine itself, the sprayer, harvester or other machinery.

“Anything we can compute on the machine and close that loop, we do,” he said. “But we also have our cloud-based solution, the John Deere Operation Center, which allows our customers to send all of their data from their machines back to the cloud through our 4G LTE network. The cloud-based solution allows the customer to back up their information. It also allows them to see what’s happening on their farm and learn what’s working well and what’s not for specific areas of their farm. It also enables collaboration.”

Farmers have a lot of trusted advisors. They help farmers make their day-in, day-out decisions and the cloud allows for data sharing and collaboration between farmers and their trusted advisors.

Inside the cab, the operator’s cockpit is just as advanced, with joystick controls and touchscreen displays providing a variety of data and real-time adjustments for variables such as crop condition, grain tank levels, machine diagnostics and performance targets. With sophisticated, automated machines, farms becomes factories on wheels.

“A farmer’s business is really a network of fields. Each one of these fields has millions of plants. Our goal is to make sure that we reach the maximum potential for each one of these plants,” Hergenreter said. “Right now, many of the farms are managed at field level. Through the technologies we’re talking about – vision sensing and software prescriptions – we can start to create plans that allow our customers to get closer to plant-level management.”

Across North America, farmers are embracing the technology and trusting the data.

Intuitive touchscreen displays put a wealth of real-time data and analysis tools at operators’ fingertips so they can monitor, adjust and learn from their machinery while moving through the field. Courtesy: John Deere/RIA

“To reach their full potential, farmers must manage each of their plants as they grow. Through technology, we’re allowing the farmers to hand off the more taxing, more repeatable actions to their equipment,” Hergenreter said. “Our goal is to allow them to generate more consistent outcomes despite all the variables, not only weather but climate, soil, and all the other variables farmers run in to day-in and day-out. Better decisions allow them to be more productive with their time, their equipment and their land. We’re trying to optimize yields.”

Ad - Make you next Shed an Entegra Shed - The best in the business - Ad

Precision agriculture is about optimizing yields while controlling costs, and promoting and maintaining sustainability. That includes protecting the environment. Protecting the land, the water and the air, and minimizing waste. Using less pesticides, fertilizer and other chemicals, decreasing fuel consumption, reducing carbon emissions, and conserving more natural resources and energy. Generations of Earth’s future inhabitants will depend on it.

The United Nations predicts the world’s population will grow from its current level at 7.6 to 9.8 billion people by 2050. The Global Harvest Initiative (GHI) expects the world’s food producers will need to increase production by 70% to accommodate that population growth.

GHI estimates that from 2005 to 2019, an estimated 58 million fewer people will be employed in agriculture, a decrease of 11% of the workforce. This presents a significant challenge for farmers trying to find skilled labor to ramp up production. The agriculture industry will need to learn how to do more with less, by adopting more efficient and sustainable production methods. Robotics and AI could furrow the way to a brighter future.

Autonomous harvesting with industrial robots

Launched in 2018, Root AI is using traditional and proprietary robotics hardware combined with sophisticated software to expand the domain where industrial robots add value. Agricultural robotics has typically involved bespoke equipment focused on a specific task or a particular type of crop. Root’s solution puts a modular collaborative robot to work on the farm and makes it even smarter with artificial intelligence.

Virgo, the robotic harvesting system, is a standard, industrial-grade cobot on a mobile platform combined with computer vision for sight, custom end-of-arm tools (EOAT) for grasping a variety of fresh produce, and onboard intelligence that enables the unit to do dexterous work in the field.

Co-founder & CEO Josh Lessing said Root is focused on the AI technology, the brain of the system. Recent advancements in algorithms, especially computer vision software for finding individual objects in complex environments, have become a game changer.

Sophisticated vision sensing, artificial intelligence and custom end-of-arm tooling enable a robot to harvest ripe produce directly from the vine. Courtesy: Root AI/RIA

“AI is the big piece of the puzzle for us. When you talk about agriculture, it’s a disordered environment,” Lessing said. “All of these computational tools began emerging, which can find things in that environment in ways that have never been done before. Simultaneously, chip producers started creating ‘system-on-modules’ (or SoMs, also called computer-on-modules or CoMs, that are complete computers built on a single circuit board), which deliver a lot of computing power to a robot without requiring an internet connection. And you can do it in a way that consumes very little power.”

Battery management and on-board computing power is very important for mobile platforms, such as Root’s Virgo, which could be working in the field. SoMs were also getting cheaper while delivering higher levels of computing in an energy-efficient manner.

“That was powerful. That allowed us to bring AI to the field,” Lessing said. “Computer vision algorithms can’t touch the physical world, they can just look at it. Robots are the bridge, and at Root we’re building machine learning algorithms that will enable robots to do physical work in complex, real-world environments.”

Machine learning for grasp planning

For a robot to grasp something and interact with it, the robot not only needs to be able to identify things in the environment, it needs to then understand how those objects in the environment relate to one another physically, the way they attach to one another, like a fruit or vegetable on a vine. Then the system needs to understand how the relationship informs the robot how to grip and remove the object from its environment. Grasp planning is an essential element.

Root works closely with a set of growers who have granted the startup access to their facilities to run product tests. The growers provide feedback on experiments and the types of features they will need for the robotic harvesting system to provide greater value. Root also is using AI and machine learning to teach their robots new tricks. Every day, the robot is collecting data. That data is used to determine smarter picking strategies, which Root uses to update the software.

A collaborative robot designed to work in compact spaces, like this indoor greenhouse, is mounted on a mobile platform and equipped with computer vision and machine learning to pick ripe produce as the collaborative robot autonomously moves along the row of plants. Courtesy: Root AI/RIA

“We keep updating the software. We keep making it smarter. When these robots are in farms next year doing work on a daily basis, updates will occur when the robots go to charge up,” Lessing said. “That’s when the data on the robot that informs better behaviors and improves performance will get pushed to the cloud, and when software updates will be downloaded to the robot. Often when you buy capital equipment, the best day is the first day. If you have the ability to do software updates based on data that you have in the cloud, and then push those updates back out to the fleet, you can have a single unit get better and better, day after day, so the best days are yet to come.”

Serving up specialty crops

Virgo is in product testing at indoor farms around the U.S. Currently, the robot is picking ripe tomatoes in large commercial greenhouses. But Root has their sights focused on other specialty crops, such as strawberries, raspberries, cucumbers, peppers, eggplants, melons, grapes and avocados. According to Lessing, these specialty crops require incredible amounts of skilled, dexterous labor. It’s been a substantial challenge to automate harvesting.

“We’re starting with the tomato, but ultimately the technology stack that we are developing is broadly applicable to all sorts of crops. Today it’s the tomato, tomorrow it’s the pepper, the next day it’s the cucumber.”

Ad - Make you next Shed an Entegra Shed - The best in the business - Ad

He said jumping from one crop to another crop will be a “simple end effector swap” to equip Virgo with different ‘hands’ that enable the robot to grasp and harvest a variety of fruits and vegetables.

“By going after one of the single biggest unmet needs in the specialty crop industry, we have folks’ attention. In fact, 30 to 40 percent of the revenue from a tomato goes straight to paying off labor. The job of being a harvester is very physically demanding. It’s a difficult job, it’s a seasonal job. The ability to source labor to meet demand is increasingly a struggle.”

The payload range for these specialty crops is well within the capability of common industrial robots. This includes a collaborative SCARA robot, the robotic arm behind the Virgo system. As far as mobility, Virgo often works in greenhouse environments that have a train rail system throughout the facility. Eventually, Virgo will be self-driving within a row of crop and will be manually moved from row to row. Lessing said this helps create a collaborative workflow.

Faster and fresher to market

Virgo is designed to help facilitate one of the newest disruptive trends in agriculture – indoor farming. Variations of the concept, including urban farming and vertical farming, are gaining ground.

If growers can pick it one day and have it on the grocery store shelf the next day, that’s the peak of nutritional value and flavor. That’s where consumer trends are headed. People want fresh produce in their diets. Delivering fresh produce consistently high in quality, nutritional value and taste, means growers need to start producing closer to where food is consumed. Right now, the logistics supply lines for fruits and vegetables are quite long and can eat into the shelf life of the product, degrading the end customer’s experience.

“Working with greenhouse growers is a rare opportunity to be part of a very disruptive and positive change in agriculture,” Lessing said. “These facilities are hyperefficient. They can use up to 90% less water than a typical outdoor farm. They are able to minimize the use of pesticides, fungicides and herbicides, and because these are controlled indoor environments, they actually use beneficial insects to attack the bad insects.”

According to Lessing, one of the indoor growers testing Virgo is producing 25 times as many tomatoes per acre as an outdoor farm. This is significant when you consider a world with less arable land. Scientists say the Earth has lost a third of its arable land in the past 40 years, and the rate of soil depletion will continue to rise if there is not a major change in agricultural practices.

“Consumers are asking retailers for food that is delivered to them through more sustainable production methods. If you can deliver a piece of produce in 24 hours to a major city, that’s fresh! If we’re able to work in those environments and deliver value, we’re a part of the future of agriculture.

“The food industry has so many challenges that simply have to be overcome,” Lessing said. “When I was growing up, my dad’s major hobby was picking the tomatoes in our backyard. He was a physician and was always very dedicated to serving his community. It was a lesson he taught me. Over my career, I’ve had a chance to learn a lot about the food industry at a business level. That experience presented a new opportunity in my life to pursue that passion, to help people in the food supply chain deliver on the promise of plentiful, reliable food.”

With their robotic system, Root has several patents in the works to help them realize this mission.

Flexing for the food supply chain

With stakes high for the world’s food producers, flexible automation helps the industry meet the challenges of a food supply chain always up against the clock. Fresh, fast and affordable leave little room for inefficiencies. Robots help growers feed the insatiable demand.

Soft Robotics Inc. CEO Carl Vause said labor shortages and inefficient processes go beyond the field. We’re not only losing food dying on the vine. We’re also losing produce and other perishable foods along the supply chain when they can’t get to market in time.

Soft-actuating gripper gently handles fresh produce of varying size and shape along the entire food supply chain. Courtesy: Soft Robotics, Inc./RIA

The Bedford, Mass.-based Soft Robotics serves three key industries: general supply chain and logistics, advanced manufacturing, and food and beverage. The grippers excel wherever applications have high rates of variety such as in e-commerce, store replenishment and delivery, sortation of produce by size and shape, meal kitting, and high-mix/high-volume manufacturing such as cosmetics and consumer goods.

Ad - Make you next Shed an Entegra Shed - The best in the business - Ad

Finding a unique niche in the food industry, the robotic grippers handle produce, protein and bakery products, all areas where fruits and vegetables, meats and fish, and raw dough and baked goods vary widely in size, shape, weight and deformability – exactly where an adaptive gripper made of soft, compliant materials comes in handy.

“When we talk about produce, we do everything from harvest to last-mile grocery delivery,” Vause said.

In the field, Soft Robotics grippers are harvesting heads of leafy greens. Traditionally, lettuce harvesting is back-breaking work. Teams of workers are hunched over rows of lettuce with machetes, slicing each head at the stem. Labor shortages make it more difficult to find workers willing to do this grueling job.

Automation comes to the rescue, saving workers from ergonomically challenging tasks and allowing them to move up to higher-value jobs on the automated farm. Combined with machine vision to locate the lettuce heads and an automated blade to separate the heads from their stems, the Soft Robotics gripper helps harvest different varieties of lettuce and other leafy greens for a U.S.-based grower.

“We’re able to grip all of the different sized heads of lettuce at a good rate and without damage,” Vause said. “That’s a huge advantage.”

Improving food cleanliness and safety

Cleanliness is an ongoing challenge for food producers. Soft Robotics has gone above and beyond to ensure its grippers achieve a higher level of food safety. The grippers use a proprietary blend of surgical-grade polymer materials manufactured to stringent standards, and meet requirements for the U.S. Food and Drug Administration (FDA 21 CFR), the European Food Contact Materials Regulation (EC 1935) and Japan’s Ministry of Health, Labor and Welfare.

Soft Robotics adheres to Good Manufacturing Practice (GMP), which is a regulatory system for ensuring products are produced and controlled according to quality standards and guidelines recommended for manufacturers, processors and packagers of food and beverages, cosmetics, pharmaceutical products, dietary supplements and medical devices.

“We use a medical-grade material that is manufactured both by our suppliers and our in-house processes to meet the GMP requirements, which is very important for traceability and to know that cleanliness has gone into every step,” Vause said. “It gives you that extra margin of cleanability and food safety.”

Confidence in a safe, reliable food supply is a basic human need. But growing populations, labor shortages and land degradation threaten sustainability. The agriculture industry will need to do more with less. Robotics and AI are up for the challenge as farms learn to function like lean factories. High-tech, clean and data-rich for a more sustainable future.

Tanya M. Anandan is contributing editor for the Robotic Industries Association (RIA) and Robotics Online. RIA is a not-for-profit trade association dedicated to improving the regional, national, and global competitiveness of the North American manufacturing and service sectors through robotics and related automation.

Ad - Make you next Shed an Entegra Shed - The best in the business - Ad