The transformer model is patented by Google. patents.google.com/patent/US10452…
@CSProfKGD Like a BatchNorm. Haven't sued anyone yet though
@CSProfKGD If the only thing left in the tank for them is to pursue patent litigation, it really would be the end of them. Like Oracle 2.0
@CSProfKGD Imagine if Google's non-existent moat affects their bottom line to the degree that Google actually enforces this patent. The ensuing legal fight would be incredible.
@CSProfKGD When visiting a major industrial lab I was asked specifically NOT to mention something was patented when I knew it was. Plausible deniability seems to be a thing?
@CSProfKGD I dont understand why go through all tthe hastle of patenting something that can never be enforced anyway.
@CSProfKGD My read is the claims are all a system comprising an encoder and a decoder. For a system to be shown to infringe would require it have all the comprising elements. Decoder only transformer LLMs like GPT/BERT have no encoder, and therefore aren't covered by this patent.
@CSProfKGD Does that mean that Google can sue OpenAI because their models use Transformer architecture?
@CSProfKGD This is why >90% of all contested patents are ultimately revoked. Thanks for padding the stats Google!
@CSProfKGD The funny question is: what if GPT-x (based on Transformer) designs a new architectue better than Transformer?
@CSProfKGD It is disturbing to witness the power of money over integrity Even if they won't sue anyone, its hard to understand why the patent-office approve this so called "patents" which are often just minor modification over existing method and common knowledge reddit.com/r/MachineLearn…
@CSProfKGD After some point in the 2010s, patenting software has been really really difficult. What you are showing here is just an application. Is there an actually patent folio ?
@CSProfKGD Thankfully Google doesn't sue companies for patents. They only use it as a defence mechanism (mostly after they were sued for Android and had to acquire Motorola for their patents)
@CSProfKGD So another non-enforceable patent like dropout and batchnorm ? 🤔
@CSProfKGD Like when God patented dna to keep Geud from jamming dna into his fleet of brumble snatcher goos… thanks God for that. But Geud’s clever sister, Glad has filled a white room with snow, slowly reversing human dna based on snow angel lithography- the dreams of unencumbered souls.
@CSProfKGD I’ve been thinking about this all week! Yikes!
@CSProfKGD As I understand it, they patented to protect themselves against trolls, because the system is broken.
@CSProfKGD Patent was filed almost 5 years ago: June 28 2018
@CSProfKGD Transformers are just a step not the destination.
@CSProfKGD Does this patent mean Google is letting people use and help develop AI until they decide to control everything? Or do they simply not want a patent war, since they also use patents of others?
