Zorse and the Hunt for the Hidden COBOL
An initiative by the Open Mainframe Project is seeking to collect training data to improve AI tools for mainframe developers.
By Neil Tardy
Zorse and the Hunt for the Hidden COBOL
An initiative by the Open Mainframe Project is seeking to collect training data to improve AI tools for mainframe developers
By Neil Tardy
Given that some 800 million lines of COBOL currently reside on production systems worldwide, the aims of the Open Mainframe Project's Zorse initiative may seem easily attainable. In reality though, it's not that simple. That's why the recently launched group is seeking volunteers to help with the time-consuming process of locating available COBOL code that can be used to train and calibrate large language models (LLMs).
AI researcher Gabriel Gordon-Hall describes the project as the product of a gradual realization. Over the past few months, as he and other members of the Zorse steering committee watched the years-old prediction that AI would evolve into the foundational layer for software come to fruition—and as they now watch AI eat everything—Gordon-Hall and his collaborators concluded that mainframe developers are, in some sense, missing out.
Basically, AI is eating the software stack, but mainframe developers are not necessarily able to take advantage of some benefits there.
—Gabriel Gordon-Hall, member of the Zorse steering committee and co-founder/CTO of Bloop AI
A Shortage of Training Data
"It was kind of just the observation that developers across fields are relying more and more on LLMs in their day jobs to write and review code," he says. "Basically, AI is eating the software stack, but mainframe developers are not necessarily able to take advantage of some benefits there. One of the reasons is that these tools don't work as well with mainframe languages. And a lot of the reason for that is that they're just not trained on much data that relates to the mainframe or mainframe languages."
Training LLMs on mainframe languages is one piece of the committee's vision for Zorse. In addition to creating a data set of open-source mainframe code that can be used to train or fine-tune LLMs, there are plans for a toolkit that would include a suite of evaluations that measure the ability of LLMs to understand and write mainframe code.
"It's in the early stages. We have a first version of a data set [available on Hugging Face] and then a first version of evaluations," Gordon-Hall says. "So it currently contains a benchmark to measure COBOL writing, like the ability of the models to solve problems in the COBOL language. We're working on adding other evaluations for other languages, too."
Of course, modernization tools that are designed to convert COBOL applications to Java are already available, including IBM's watsonx Code Assistant for Z. And Gordon-Hall himself is co-founder/CTO of startup Bloop AI, whose flagship product is described as an AI agent for COBOL modernization. Again though, he points out that increased training on COBOL and other mainframe programming languages would simplify the processes of maintaining COBOL code or mapping it to Java.
Explore the Latest Solutions With TechChannel’s Solutions Directory
Discover the tools and strategies shaping the future of technology with the 2025 TechChannel Solutions Directory. From AI to cybersecurity, this resource is designed to help you stay competitive and connected with the tech community.
If you're maintaining code, a huge amount of your job is understanding where the logic is that does something, or what a particular bit of logic does that somebody wrote years ago that might be broken.
Gabriel Gordon-Hall, member of the Zorse steering committee and co-founder/CTO of Bloop AI
A Deeper Understanding
"If you're reading some COBOL code and you want to know what it does, people just use language models like ChatGPT or these LLM-based tools to do that: ‘What does this do? Break it down for me in five simple steps’ or something like that," Gordon-Hall explains.
This process of understanding is critical to maintaining code that may have been written long before today’s young engineers were born.
"If you're maintaining code, a huge amount of your job is understanding where the logic is that does something, or what a particular bit of logic does that somebody wrote years ago that might be broken. People are already using language models to help with those kinds of tasks," Gordon-Hall says.
To move forward, Zorse is seeking both contributions of COBOL code and volunteers to handle day-to-day tasks. Those with interest are encouraged to register and post on the Zorse Discord page.
Uncovering the Code
While the Zorse committee has already collected some COBOL code—primarily from hosts such as GitHub and GitLab—online sources with more plentiful amounts of COBOL aren't as easily scraped. Think of megabytes of code contained in .zip or .tar files. Gordon-Hall notes that for years Medicare pricing data from the US Health Department was written in COBOL.
"There's always been data that's slightly harder to get," he says. "We have to be a little bit more in the know."
That said, a source of particular interest to the Zorse committee is mainframe developers and programmers themselves. Do you have a long-unused hard drive with COBOL or even Rexx or JCL? While these cases would be assessed individually, Gordon-Hall stresses that Zorse would take measures to ensure that only legally available code would be accepted, and that any identifying information would be scrubbed prior to training.
"We want to grab as much as we can that's already on the internet. But then to be honest, the ideal cases would be where people come forward with some examples of code which are not on the internet, which they'd be willing to donate," he says.
In addition to code, Zorse is seeking time. Gordon-Hall envisions master’s or PhD students having interest in working with the training models.
There’s always been data that's slightly harder to get. We have to be a little bit more in the know.
—Gabriel Gordon-Hall, member of the Zorse steering committee and co-founder/CTO of Bloop AI