Featured AI2er: Patrick Beukema
Patrick Beukema is the Lead ML Engineer for Skylight
What put you on the path to your current role?
After my son was born, I grew increasingly concerned about the state of the planet, both now and when he’s grown up. At the time, forest fires were wreaking havoc on the west coast. While those may not be the greatest environmental threat that we face, the blanket of smoke covering Seattle for the better part of a month was a stark reminder of how poorly we are treating the planet. Over paternity leave at my previous company, I became increasingly motivated to work more directly on sustainability and fighting environmental degradation. AI2 happened to reach out around that time, and I interviewed for a role at the Applied Science and Technology (AST) organization. During those interviews, I met a group of passionate people dedicating significant time and resources to help solve difficult problems in the environment and conservation. They were motivated, smart, and skilled, especially in machine learning and artificial intelligence. Most importantly, they were making tangible progress. I had spent the better part of my life searching for impactful work that could successfully scale. After meandering my way through academia, a startup, and big tech, I finally found that at AI2. Our (Skylight) team’s work and that of our sister teams (Wildlands, Climate Modeling, and EarthRanger) are AI2’s effort to create a more sustainable future and a healthier planet. I joined Skylight as head of ML, and it has proven to be a dream role.
What’s the most surprising thing that has happened with your work at AI2 recently?
Working at AI2 has given me new hope in our ability as a society to solve massive and seemingly intractable problems like global warming or marine conservation. Hope in the context of today’s dire headlines is hard to come by, so I want to share a few reasons why. At Skylight and within AST, we have been increasingly introduced to world experts, spanning academia and industry, who recognize that we are making progress but could benefit greatly from their help. We are a small team determined to change the course of human impact on the environment, but there are many areas in which we lack necessary technical depth, like dynamical systems and reinforcement learning. Considering the demand and astronomically high salaries associated with AI right now, it’s encouraging that these experts are choosing to spend their time and energy helping environmental causes. What is also encouraging is that the most skilled experts I’ve met have surprisingly low egos and a deep appreciation for teamwork, collaboration, and genuine impact.
How does working at a non-profit differ from your previous work?
Working here is profoundly different than any place I have ever been previously. AI2 is uniquely privileged: we benefit from a large endowment that enables us to work freely on big problems insulated from the unforgiving and competitive commercial marketplace. But this is a double-edged sword, because it also means there are no self-correcting market forces that will kill a bad idea or promote a good one, yet the stakes are high. With a conventional product, the goal is typically to make money, and therefore it’s pretty easy to measure how you are doing. If your goal is to improve science, or to make better predictions about climate change, what is the metric that should be used to quantify success? And if one cannot select a metric, then how do you measure the causal impact of an intervention (such as the Skylight platform)? Measuring impact, and then tying those measurements to our individual objectives and day-to-day activities, such that there is incentive alignment across our team, our users, and all relevant stakeholders (including the planet), is not trivial. But it’s not impossible either. And there are (albeit) imperfect metrics that we already can anchor on, such as securing external (non-AI2) funding, ensuring our models are on par or exceeding the best models in the industry, and doing so with fewer resources.
What are you looking forward to with your work in the coming months?
Skylight AI is only in its nascent stage. Later this month, we will be adding a third real-time satellite computer vision service for vessel detection using the Sentinel-2 optical imagery from the European Space Agency. That service will complement two existing computer vision services covering Sentinel-1 (Synthetic Aperture Radar), and VIIRS (nightly vessel detections).
In addition to computer vision for satellite imagery, we have spent a large portion of this year building an entirely new foundation for GPS-based vessel monitoring and seq2seq behavioral classification that is about to be released. At any given moment on the high seas, there are millions of vessels all around the world doing many different things, from sailing and transporting cargo to more nefarious actions, like illegally dumping oil or fishing in marine protected areas. Our users rely on our models to distill the massive amount of GPS data collected by satellites, comprising hundreds of millions of data points per day, into usable intelligence. We are in the process of fine-tuning and testing a new collection of deep neural networks designed and trained in-house that autonomously reviews all of the available information (GPS, vessel photos, wind and wave data, satellite imagery, and more).
Our goal is to provide highly reliable, precise, and real-time classification of every single vessel’s behavior on the planet. Despite the superficial differences between GPS sequences and natural language sequences, the overall model architecture and training strategy will be familiar to anyone building LLMs. We started by building an entirely new annotation platform to robustly scale our training data and hired maritime intelligence experts to annotate millions of vessel GPS points. We created new neural net architectures specialized for maritime intelligence that can run at global scale in a streaming context. These nets are called ATLAS (AIS Transformers with labeled active subpaths) and the entire classification system, which is composed of many ATLASes, is called Atlantes. Atlantes will soon power a variety of events in our platform and will be freely available to all of our users early next year.
We’re also developing methods to leverage geospatially focused large language models (LLMs) to broaden our reach and user engagement. At this year’s hackathon, a strong collaboration with Aristo’s (one of AI2’s NLP/ reasoning teams) enabled us to create an initial version of this model within a 24-hour timeframe. Over the coming year, we’re excited about rolling out a natural language interface to enhance accessibility for all our users.
We are looking forward to traveling to New Orleans this year for NeurIPS and the inaugural Computational Sustainability workshop. We will be presenting a collaborative paper with our colleagues from PRIOR, AI2’s computer vision team, titled: “Satellite Imagery and AI: A New Era in Ocean Conservation, from Research to Deployment and Impact”. We consider this to be just the beginning of deeper synergies between Skylight and AI2’s world-class research teams and look forward to more fruitful and impactful collaborations to come.
What is your favorite thing about working at AI2?
At a lot of public companies, there is a considerable amount of public talk about how DEI is a central tenet of the workplace. My professional experience prior to joining AI2 is that those pronouncements are usually PR doublespeak to wokewash morally bankrupt decision-making for the purposes of good (and vapid) press. AI2 is not perfect, but from what I have seen, the culture here takes diversity of thought and background very seriously and is constantly encouraging growth. To offer one short but simple example: an intern over the last summer recommended a podcast on AI and education but added the caveat that it was a “manel”. I had never heard that term but its meaning is unsurprising: a manel is a panel consisting of all men (here’s an article on why they suck). My wife and I use that term regularly now (it comes up far too often, especially in tech).
What do you consider the most underrated activity or place in Seattle?
The biking here is world-class, every kind — gravel, mountain, road. Travel in any direction and you’ll find great trails, many still unmapped. The Olympic mountains, the Salish Sea and the Puget Sound, lake Washington and the Cascade mountains. Seattle is smack in the middle of a double water mountain sandwich (my wife’s epithet for Seattle).
What new hobbies or activities have you tried lately?
Over the last couple months, I picked up portrait photography. I wanted to capture my now 2 ½-year-old son’s heart-melting expressions, laughter, and joy, and our family’s memorable moments. There are a lot of photographers in the office, especially on the wildlife team, and their learnings from professional wildlife photography has translated well to capturing the fleeting moments of toddlerhood.
Learn more:
- Many of our computer vision models are open-sourced. See https://github.com/allenai/vessel-detection-viirs and https://github.com/allenai/vessel-detection-sentinels
- We host a monthly environmental AI seminar and many talks are publicly available on AI2’s YouTube channel.
- Recent talk on VIIRS at NOAA’s 5th AI workshop.
- Our computer vision work will be posted to arXiv.
Follow @allen_ai and @SkylightMarine on Twitter/X and subscribe to the AI2 Newsletter to stay current on news and research coming out of AI2.
