UND researchers, startup HCI partner to advance autonomous technology in UAS

UND’s Research Institute for Autonomous Systems brings fresh ideas to bear in collaboration with startup

Jordan Krueger (left) and David Johnson, two longtime friends, were both heavily involved in the Research Institute for Autonomous Systems’ work with Minnesota startup Hughes Company Innovations. Since 2019, Krueger has been a project manager with RIAS. Johnson got involved by means of his senior design project in computer science. Image courtesy of UND RIAS.

Partnering with a startup company out of Minnesota, UND’s Research Institute for Autonomous Systems has shown its ability to help entrepreneurs take an idea from conception, to fundamental laboratory research, and through to commercial viability.

In early 2020, such a partnership between UND and Jim Hughes, founder and CEO of Hughes Company Innovations (HCI), took flight as the two entities began a multi-month project. Ultimately, the project would successfully deliver a unique, autonomous, aerial data collection system for a multinational company’s industrial sites, sites that cover a land area of more than 100 square miles.

Jim Hughes

According to Hughes, the scalable system combines airborne and ground-gathered data to provide unique insights and drive benefits such as greater safety, fewer demands on personnel, better environmental performance and extended uptime for machinery, infrastructure and similar industrial assets.

“Using unique sensors around different parts of an operational worksite, we aim to provide data to the customer that really helps them achieve better business outcomes,” Hughes said. “Our work is about solving real business problems using technology and innovation, and applying them in an effective way.”

The collaboration has been a chance for RIAS to not only work with a young startup such as HCI – a tenant of UND’s Center for Innovation and also a member of RIAS’s Corporate Board of Advisors – but also to research the ways in which autonomous systems can improve large-scale operations, such as resource extraction, through enhanced safety and efficiency, said Mark Askelson, executive director of RIAS.

“One of our great opportunities for growth and further development of market-ready capabilities is through collaboration with industry,” Askelson said. “Our work with HCI is a great example of that – bringing together RIAS’s and HCI’s capabilities in order to solve problems.”

Safe, effective and efficient through autonomy

This challenging project was one of Jordan Krueger’s first experiences with the research institute as a newly hired project manager. But the recent UND graduate was no stranger to skirting, surmounting or sometimes just passing through hurdles: he joined the RIAS team after helping lead the Collegiate Drone Racing Championship effort during his senior year.

Jordan Krueger

“There isn’t another job like it in the world, and I’m so lucky to have it,” said Krueger of his RIAS position. “When I got into it, I didn’t fully know what it meant to be a ‘project manager’ here, but the work we ended up doing with HCI was right up my alley.”

As it turned out, Krueger’s skills as a drone pilot, fresh out of UND’s UAS degree program, would be just as important as his ability to keep the University’s side of the effort on schedule. In addition to giving him an understanding of commercial UAS regulations, namely the Federal Aviation Administration’s Part 107 ruleset, Krueger’s time at UND (including as a teaching assistant) taught him a great deal about both remote sensing and the challenges of conducting observation flights.

“I had done a lot of work, both as an undergraduate and with RIAS, with sensors such as electro-optical, thermal and multispectral cameras, as well as LIDAR and sonar sensors that can be used on UAS,” Krueger said. “But one of the biggest things for us was figuring out how to schedule flights during our short on-site window, to ensure we gathered every piece of data available from all sensors and at all times of the day.”

“In addition to our ‘ideal’ schedule, we built out detailed plans for any weather delays,” Krueger continued. “It was difficult to keep everything straight, but was extremely beneficial to the project.”

These pieces, joined with the research power of the RIAS team, helped provide real-time, actionable intelligence to HCI’s customers that increased overall awareness of worksite conditions. Such intelligence lets customers more precisely observe critical components of their operations, boosting the odds of spotting problems before failures occur.

And by being able to see and use the power of autonomous operations, HCI’s customers can sense the safety benefits, efficiencies and productivity improvements provided by not only UAS, but also other autonomous systems that can collect and process data, said Askelson.

Hughes agreed. “What’s unique about UND is the work that they have done in UAS research, such as around beyond-visual-line-of-sight operations,” he said. “In working with RIAS, I’m looking at how we can take these amazing aspects into the market to solve problems and create better outcomes for industries.”

Creating intersections for business and academia

Following 20 years as a business consultant and managing director, helping some of the world’s biggest and most successful companies, Hughes wanted to use his experience to launch his own company – one that is reshaping how the business world and academia interact in an age of digital disruption.

Krueger’s flight skills and UAS know-how came to the fore in working with HCI, which represented one of his first projects in working for the UND-based research institute. Image courtesy of UND RIAS.

In his words: “How can we use the market to make sure that we’re focusing on what’s innovative, to make sure that we’re focusing on solving problems that actually matter?

“Then, how do we do that in a way that engages students and faculty in a way that’s a learning process for all involved?”

It was this thought process that led to David Johnson joining the collaborative effort. As a UND senior majoring in computer science, at the time, Johnson was in the market for a good senior design project. As engineering students learn, these projects represent academic-year-long efforts to bring one’s cumulative knowledge to bear – a course-based experience of what it’s like to work in the “real world.”

As a longtime friend of Krueger’s, Johnson heard about HCI’s collaboration with RIAS and got involved as a means to fulfill his degree from UND’s School of Electrical Engineering & Computer Science.

“David’s instructor knew this was an experience that David wouldn’t be able to get anywhere else,” Krueger said.

In fact, Johnson developed a ground-based measurement that helped with the monitoring of the worksite at large. Almost from scratch, the senior came up with a system to read water table measurements in a number of locations; relay those measurements to a single unit on the ground; then beam those sets of data to a UAS as it flies by.

With the UAS already flying past to collect data from other places on site, Johnson’s coding work was key in allowing the overall system to gather even more data without needing a separate retrieval effort, such as physically going to a ground station to read an SD card. On top of that, Johnson created an operational manual to guide the installation and operation of the equipment he created, as he wasn’t able to be on location during the operational period of the project.

Now graduated and employed in the aerospace industry, Johnson said he is glad he took on the challenge. Working in the fast-paced environment of HCI’s project was certainly a new experience, he said, and he learned valuable lessons in teamwork and communication that contributed to the start of his professional career.

“I had to provide updates to my class for the senior design aspect, and also to Jim [Hughes] and the RIAS team, which were two different levels of detail to consider,” Johnson said. “I was able to learn a lot about how to communicate my progress and/or holdups in both of those scenarios.

“With HCI being a new company, it was cool being able to help Jim in that overall effort of figuring out the services that could be offered to his customers.”

Shared vision for R&D innovations

In addition to RIAS’s role in creating solutions through autonomous technologies, whether for individuals, governments or industries, RIAS has an important role to play in workforce development, Askelson said.

Mark Askelson

“By doing this research and getting our students involved, we are helping produce that next generation of skilled personnel – people who can go out into the world and create solutions that communities need,” Askelson said.

The partnership with HCI exemplifies why UND embraces and seeks to expand industry-university partnerships, said John Mihelich, vice president for research and economic development at UND. The University can help companies solve research and development problems because it can draw on the expertise of faculty and students across many disciplines.

John Mihelich

“Through the collaborations, we expand our research capability, apply our research innovation to problems that have commercial applications, and, when suitable, hone our responsiveness to market needs,” Mihelich said. “Equally as important, we engage students in the process. Students gain valuable experience in research and the industry environment, both of which will open up career pathways for them and facilitate UND’s role in developing the future workforce.

“The partnership with HCI, at its root, stemmed from a shared vision to integrate UND’s education and research missions with industry-based expertise through a purposeful collaboration. I think the team implemented that vision quite well.”

Through student experiences such as Johnson’s, and collaborating with research entities such as RIAS, Hughes sees the potential for HCI’s success to have healthy implications for increasing UAS research opportunities and shaping education by means of the market.

In an interview with UND Today, the HCI founder said he wants HCI to build on its impact to the University. “As big as HCI gets, I want UND and RIAS to be part of that.”