I plan on entering ML research in the near future, something I'd planned on for a while; I'm hopefully going to do a Master's next year in AI/data science. Rather than AI interesting me merely for the obvious (the money pouring into it!), I'm far more interested in the latent capabilities of models as they relate to language. Language is, after all, quasi-symbolic. It has components that are evidently logical (hierarchical structure, semantic composition, how words have specific, abstract necessities in different languages, such as how some verbs are transitive and others not), but components that are similarly gradient (within the possible space of linguistic "rules", wherever they live, either in stored constructions or some innate grammar, language is remarkably flexible).
Representing language as a cognitive system is still a challenge to some degree; humans process and parse language incrementally (we predict what comes next within a stream of information), and some sentences sound stranger than others. A good example: garden path sentences. "The horse raced past the barn fell." Incrementally, "the horse raced past the barn" seems to be a normal sentence. The addition of "fell" throws in some confusion. But read it again: "raced" here isn't an action the horse is undertaking but rather receiving. The horse was raced past the barn. Or, "The complex houses married soldiers and their families." This one's more obvious: you first read "complex houses" as an adjective and a noun, but in reality, "the complex (noun) houses (verb)".
In the end, language is fundamentally a human capability, and to understand linguistic structures, you must understand how humans think, the limitations of the human brain, how people both produce and comprehend language and how it's represented within the brain. Where do neural networks fit in? Well, they've found some success and failure as proxies for certain facets of language representation like syntactic surprisal (Arehalli et al. 2023).
I think that if you're interested in the debate on how neural networks can assist in understanding how humans naturally produce, represent, and understand language, I recommend looking at the "they pretty much are models, pack it up" perspective (Piantadosi 2023), the "Yeah, they can't tell us much" perspective (Kodner et al. 2023), and the middle of the road approach, where I stand, in Futrell and Mahowald (2025), and their more recent 2026 paper "You Can't Fight in Here! This is BBS!". It's a really good look at it.
One of the most interesting things I've recently seen is the fact that neural networks may be trained on strings, but actually represent language hierarchically! Moreover, they do show sensitivity to agreement (subjects and objects in a sentence). McCoy (2026) says modern neural networks implement a "quasi-symbolic" system, where symbolic operations are approximated. I think that's awesome!
Their use in testing linguistic hypotheses has similarly been tested with some success: Przepiorkowski and Patejuk (2025) argue for a symmetric account of coordination given the dependency minimization hypothesis, testing it out with a neural dependency parser on the Penn Treebank. But this is pretty preliminary and I can't say too much about that as I'm not the most aware.
This is all to say that this work will soon be my bread and butter, so I have very specific things to say about it. One thing is regulation. I've seen folks talk about "regulation", but what regulation? It's not something to leave to legislators to figure out; they don't have the best track record for understanding how modern technology works (cough SOPA cough), and the recent ill-defined and criticized Bernie Sanders bill to illegalize "superintelligence" is just more crap. Are there problems with the technology? Sure, sure as any other technology. But "regulation" calls pretend as though government is anything other than a small conglomerate of individuals with power, and not an omniscient and benevolent organization that knows best for us.
The real interesting research is happening in places where people don't think to look. That's just how it is. I personally wouldn't want a bunch of old moralists to tell me what's best for me while being unable to use Bayes' Theorem. I think that's a fair enough outlook.
