#OpenAI's ChatGPT is ready to become a lawyer, it passed a practice bar exam! Scoring 70% (35/50). Guessing randomly would happen < 0.00000001% of the time
I wonder if the training data included any of these questions?
@pythonprimes Awesome. We have been using GPT3 (and some other models) to create a legal assistant in Word with spellbook.legal Really impressive results but didn’t know we were in pass the bar territory!
@pythonprimes Awesome. We have been using GPT3 (and some other models) to create a legal assistant in Word with spellbook.legal Really impressive results but didn’t know we were in pass the bar territory!
@pythonprimes An interesting exercise would be to prompt it as such: You are an expert attorney and are asked to give legal advice. {question}? And then see if it can generate the correct answer instead of selecting/guessing
@pythonprimes What would be the chances for someone that never studied, but has unlimited access to the internet? Just to put things in perspective
@pythonprimes @AMLane_au I’ve got an idea for when you go to Barrister school next year!
@pythonprimes can you share the code for this analysis so it can be replicated?
@pythonprimes wow, this is cool! I thought contract law would suffer but not this.
@pythonprimes It is a doctor too. And an engineer , Astronaut ...
@pythonprimes I still personally think this is too high-risk to delegate directly to AI But, I wouldn't be surprised if a startup primarily uses AI then gives the output to a lawyer to make sure there's no mistakes. It would still increase lawyer productivity.
@pythonprimes The real value proposition here is medicine, especially at the GP level. Going to be fun watching the AMA get involved.
