We just reported about the Linux 5.12 changelog with a focus on Arm, MIPS and RISC-V targets on Tuesday, and at the time, the expectation was a delay of about one week after Linux 5.12-rc8 was outed on Sunday, April 18.
But Linux 5.12 could be further delayed due to shenanigans from two Ph.D. students doing a research project on open-source vulnerability at the University of Minnesota. This was announced by Greg Kroah-Hartman on the Linux kernel mailing list.
Commits from @umn.edu addresses have been found to be submitted in “bad faith” to try to test the kernel community’s ability to review “known malicious” changes. The result of these submissions can be found in a paper published at the 42nd IEEE Symposium on Security and Privacy entitled, “Open Source Insecurity: Stealthily Introducing Vulnerabilities via Hypocrite Commits”
So their work at to be reverted with 190 reversions so far. It also means future commits from the University of Minnesota will be rejected from mainline Linux by default.
You’ll find the research paper hosted on Github. The research paper mentions targets for “hypocrite commits” include large projects that accept commits from third parties including the Linux kernel, Android, FreeBSD, Firefox, Chromium, OpenSSL, and others. But it appears their Proof-of-Concept experiment only targets the Linux kernel using UAF (use-after-free) vulnerabilities as they identified over “6K immature UAF vulnerabilities in the Linux kernel that can potentially be turned into real vulnerabilities”.
The paper also implies they did so ethically and informed the maintainers as soon as one of their patchsets was confirmed.
Our goal is not to introduce vulnerabilities to harm OSS. Therefore, we safely conduct the experiment to make sure that the introduced UAF
will not be merged into the actual Linux code…
Once a maintainer confirmed our patches, e.g., an email “looks good”, we immediately notify the maintainers of the introduced UAF and request them to not go ahead to apply the patch.
…
All the UAF-introducing patches stayed only in the email exchanges, without even becoming a Git commit in Linux branches. Therefore, we ensured that none of our introduced UAF bugs was ever merged into any branch of the Linux kernel, and none of the Linux users would be affected
They also wrote they point out the bug to the maintainers and provide the correct patches. But after reading Greg KH email I don’t think he shares the same feeling. It could be due to miscommunication, we just don’t know at this point.
Evaluating the security of open-source projects is a commendable initiative, but testing is certainly an issue, as it can’t be done by informing maintainers that you’re about to introduce “hypocrite commits” in advance. I don’t know the right answer, but if you feel you do, please feel free to comment below.
Thanks to dgp for the tip.
Jean-Luc started CNX Software in 2010 as a part-time endeavor, before quitting his job as a software engineering manager, and starting to write daily news, and reviews full time later in 2011.
Support CNX Software! Donate via cryptocurrencies, become a Patron on Patreon, or purchase goods on Amazon or Aliexpress
I think it is ok to ban University of Minnesota for 10 years as a standard procedure when trying to insert bugs/backdoors into a project. Malicious developers are just as bad as the incompetent ones. Why keep them around?
They were not malicious.
They were testing defenses, which is important in order to catch actually malicious people.
The way they went about it was totally wrong. All of the thousands and thousands of reviewed patches already contain the data they needed. There are going to be plenty of commits that add memory errors (I added one myself during 5.12) and a lot in the exact class they were targeting of commits that claim to fix memory errors but actually created new ones. They could have made their hypothesis and then written the analysis tool they claimed to have to go back through the git history and kernel lists to find commits and patches that fit their hypothesis… Read more »
Totally agreed, it’s clear they were seeking immediate fame through a lot of buzz. It’s a success, I didn’t know that this university existed, now it’s known all over the world!
the correct way would haven to approach someone like GKH, LT, … and figure out a procedure. it’s not that GKH or LT are the maintainers for the entire kernel. but if GKH, LT, … aren’t open for such research shame on them.
You can’t do this research by telling Linus and/or Greg that you’re going to send dodgy patches on purpose and to not accept them from sub-system maintainers if they end up in pull requests.
Do you think anyone would trust them again if they let a bunch of researchers use them as guinea pigs? Imagine you spent your free time reviewing patches, thought everything looked in good shape and then when you sent your PR to Linus he said “hah! you let through one of the dodgy patches!”. For some people that would be decades of trust down the toilet.
American hackers trying to destroy the largest international community piece of code :(. Could they be related to some of their government stance, that try to add security breach everywhere as possible?
Although it comes from an American university, the nationality(ies) of the students is not clear.
Name them: Qiushi Wu and Kangjie Lu. Kangjie Lu is an assistant professor.
I had this suspicion, and it turns out their nationalities are exactly as suspected!
The person who sent the “slander” message to Greg KH was Aditya Pakki.
“I didn’t know that this university existed, now it’s known all over the world!”
They kneeled on the neck of the Linux kernel.
Yesterday I read alot of the thread on lkml and to be honest either those guys are extremely incompetent or extremely dishonest or any combination of both…
You convinced me. Unless they found some new method disgusting the malicious code, test with the same thing has been repeatedly detected, is wasting everyone’s resource.
Honestly I think they where partnered with some government entity. Intentionally adding any bug is wrong and ethics is the most important thing about security and IT. Please just ban that college altogether for the future of Linux
They fully knew what they did and did it again. I mean come on people like this is why Linux is so broken.
I don’t believe there is anything ethical about the actions of either side. An email stating the premise of the attack would have sufficed. The demonstration of it has eroded confidence and cast doubt on ALL past commits from anyone. The unilateral “banning” of an entire organization is collective punishment[1] and affects the innocent far more than the guilty. I doubt the actors were the first, nor will they be the last, to implement this type of attack. I would rather hear how Greg intends to prevent this from happening from non-University of Michigan contributors. I fear the real answer… Read more »
I don’t think this will cast all past commits into doubt. I think it will make people suspect one off patches, huge batches of machine generated patches, patches from totally unknown people, patches with bad/vague commit messages etc. If it stops people like that guy that sent out hundreds of small typo patches that were not formatted correctly, added new mistakes etc etc then I think that’s actually a good thing. I’m pretty sure Greg’s ban isn’t a permanent ban and more like “until we can be sure otherwise we should not accept anything from these guys as they can’t… Read more »
And anyway from day one the principle of contribution has always been based on trust and reputation: if you appear trustable we can work with you, if you betray us, you ruin your reputation. This is as simple as this. There’s no reason for this to change as it remains quite solid. OK these guys managed to introduce crappy commits. A few weeks later they’re reverted, end of the story. Linux will continue to progress. However when these guys leave the university, they will have some fun trying to find a job. They’ll probably start a “security” startup like all… Read more »
> However when these guys leave the university, > they will have some fun trying to find a job. I personally don’t wish any long lasting consequences on them. Everyone should have a chance at redemption and all that.. It would have been nice if they had replied to Greg properly, admitted they had messed up and then helped to clean up the mess they caused. Right now it seems like the community has to wade through the series of 100+ reverts Greg sent so even more time is getting wasted. I guess their university has banned them from touching a… Read more »
> I personally don’t wish any long lasting consequences on them. > Everyone should have a chance at redemption and all that.. Sure, I agree, at the same age many of us have been doing stupidities like this. I’m not proud of my old findings on bugtraq 25 years ago pissing off people with zero day exploits by reporting issues without helping to get them fixed :-/ > It would have been nice if they had replied to Greg properly, admitted > they had messed up and then helped to clean up the mess they caused. Yes that’s the real… Read more »
The CIA / NSA pays contractors to do this kind of dirty work since long, in hardware, OS and popular software. All spectrum dominance is their 1984 motto, also for the US itself. Leader of the free world my *ss.
Sorry, I can’t blame Greg here.
https://news.itsfoss.com/hypocrite-commits/
“This is not ok, it is wasting our time, and we will have to report this, AGAIN, to your university…”
The commits weren’t as stealthy as they thought. The University of Minnesota researchers had a chance to deescalate but started talking about “slander”.
U of MN students can probably get around it by using a different email address, or get a little history lesson if they find out they’re blocked. U of MN can negotiate if they want the ban lifted. Nothing wrong with this punishment.
Well this university seems to have quite a sloppy ethical commission and no guidelines when it comes to public communications, so why not banning them until there is an official statement from the university (no not from some dodgy students/professors)
“I fear the real answer is “he can’t”.”
Always fix the problems at the root.
Banning the university won’t solve anything.
Banning the university actually will be very effective to stop similar incidents, not just from University of Minnesota but all other institutions doing the similar thing.
The owners hated people pointing out there was not enough life boats on the Titanic.
I’m pretty sure the fact that there aren’t enough reviewers and every reviewers load is far too high has been brought up over and over again. It’s why there are efforts to get new people involved by hand holding them through submitting their first “fix typo” patch and there are so many bots trying to break the kernel to find human errors. These guys highlighted the issue maybe but it’s not like there was secret plot to hid the fact that to maintain quality at the current pace it’s not just about big vendors throwing code at LKML anymore more.… Read more »
Remember professionals designed and built the Titanic and said it was unsinkable..
All the reactions now are doing is putting people off testing and reporting methods of attack.
If you pull the blanket over your head the bogeyman cannot see you, to get you. Theory of safety and security.
>All the reactions now are doing is putting people off >testing and reporting methods of attack. Oh shut up. No one has said the research is bad. Everyone has said the way they went about it was stupid and unethical. They could have proved their hypothesis without interacting with the LKML. There is decades worth of data out there for them to work from. They could have analyzed data from the LKML and then told everyone on the LKML that they found it’s possible people have snuck in bad code in the past and they would have be praised for… Read more »
pretty nice explanation of what happened:
https://www.zdnet.com/article/greg-kroah-hartman-bans-university-of-minnesota-from-linux-development-for-deliberately-buggy-patches/
The university issued a public statement:
https://twitter.com/UMNComputerSci/status/1384948683821694976/photo/1
It looks like they did this for months.
https://twitter.com/kengiter/status/1330649966214340608
That’s probably why Greg KH said enough is enough…
His rationale is analogous to a medical professional adding poison to a cure just to prove it can be done and then administering an antidote right before the patient dies. Then, he expects everyone to thank him for showing a vulnerability exposed by the trusting patient.
#Unethical
Well he doesn’t hand out the list of his patients, neither does he give the antidote himself, he just mentions the drug store around the corner will know what to do.
For months and it seems also past the communicated deadline of the “research” that’s why certain people on the lkml seem to suspect the clean up is part of another “research” campaign going on… Very fishy, very weird how they answer or rather not answer to direct questions on the list…
Ignoring the controversy, I found the paper to be very interesting and a worthwhile read.
Just delete it.
The person who basically accused Greg KH of slander is Aditya Pakki, different from the paper’s 2 authors, Qiushi Wu and Kangjie Lu. He appears to be a 2nd year PhD candidate.
Wonder if a grad student was “talking out of turn”…
If AI would improve quality of reviewed code (commits), then we (as global society) are prepared to delegate setting up human readable code sources to AI as well (only defining [syntax,semantics,array] of results anymore)?
If this would be a discussion between compilers and machine code, why an erroneous input occurred would be cleared in micro seconds and effective work on improved code structures would get more attention?
This was a study about human interaction?
AI doesn’t exist, yet. It’s just a marketing term for ML and pattern matching
If you think otherwise, please point me in the right direction
If you combine it with Blockchain that counts…
if its about implementation of logic’s structures, we probably could do a lot more of automatically organized source code, but e.g. NDA’s or other items from political, law related or social ranking dependent influences are less likely being easily to obey on statistical AI’s training or ‘in some way of empirical’ AI approach? There are meanwhile www search engines rules that are weighed on their semantic probability, but AFAIK not transparently to users knowledge or even to monitor on influence to search results. If AI’s would suggest variants for human reviewers to choose from, that could be approach towards (kind… Read more »
having a button on search engine pages, for including or excluding usage of (semantic) AI algorithms for interest related results of their users, this would be an empirical, user concerned idea of weighing that interest on company’s side and showing difference in results on users/researchers side without compromising companies internal knowledge?
I think there are two different things here and people focus on the wrong one. OK, it was a mistake to do that in that way, that’s the one thing. The most important thing though is that now it’s clear that anyone can push malicious code and create backdoors in the kernel. It doesn’t even have to be a single commit, those backdoors can be a series of malicious patches that can be pushed with many months difference in between, but combined are creating a backdoor. So, it’s more important to keep the later rather the fact that those “researchers”… Read more »