* Background is here if you need it. Daily Mail…
A group of University of Minnesota Law School instructors administered popular artificial intelligence bot ChatGPT four law school exams that it took alongside real students, and discovered that the bot was a C+ student on paper.
The ChatGPT bot is currently considered the world’s most advanced example of generative intelligence.
The instructors were curious about ways the bot might be used both to help students cheats and professors teach, so they decided to test its abilities.
The bot ended up with a ‘low but passing grade’ in Constitutional Law, Employee Benefits, Taxation, and Torts. All told, it wound up with a C+ average.
According to the paper, the bot came in 36th out of 40 students in Con Law, 18th out of 19 in Employee Benefits, 66th out of 67 in Tax, and dead-last out of 75 in Torts.
* A few days ago, I asked ChatGPT for some help with the blog…
Q: Summarize today’s news about Illinois government and politics.
A: I’m sorry, I don’t have the capability to provide you with today’s news as my training data only goes up to 2021 and my knowledge cut off date is 2021.
To which a friend replied in a group chat…
Oh my God, is ChatGPT Mike Madigan?!?
Heh.
* Yesterday, I thought I’d ask ChatGPT a question that might be able to provide us some humorous blog content…
What would Abraham Lincoln say to Illinois Gov. JB Pritzker?
The answer was really weird. Here’s a brief excerpt…
The process of creating a podcast involves several steps:
1. Idea generation: The first step in creating a podcast is to come up with an idea for the show. This could be a topic you’re passionate about, a hobby you enjoy, or a problem you want to solve.
2. Research: Once you have an idea for your podcast, you’ll need to research the topic to ensure that it’s something people will be interested in listening to. This could involve looking at the competition, identifying gaps in the market, and talking to potential listeners.
Um, what?
* But I went back last night and that answer had mysteriously disappeared and was replaced by another one…
It is impossible to know exactly what Abraham Lincoln would say to JB Pritzker as they lived in different times and contexts. However, based on Lincoln’s speeches and writings, he likely would have emphasized the importance of preserving the Union, promoting equality and justice, and working for the betterment of all citizens.
Not great, but at least it’s on-topic.
* A suburban legislator I talked with uses ChatGPT to draft some letters and says it’s like having a world-class chef helping in the kitchen…
It’s the biggest technological innovation since at least the Internet. Will reorient almost every sector of the economy over the next 5-10 years. I don’t know what the policy implications are yet but I’m certain it will raise novel legislative issues we will need to deal with.
* Not long after that conversation, I asked ChatGPT to write a letter to an Illinois legislator’s constituent defending the state’s new assault weapons ban…
Dear Constituent,
I am writing to address your concerns about the recently passed assault weapons ban law in Illinois. I understand that this is a highly debated issue and one that many people feel strongly about. However, I firmly believe that this law is necessary for the safety and well-being of our community.
First and foremost, it is important to understand that this law does not prohibit the ownership of all firearms. It specifically targets the possession and sale of assault weapons and high-capacity magazines, which have been used in numerous mass shootings in our country. By limiting access to these types of weapons, we can reduce the risk of mass shootings and other gun-related violence in our state.
Additionally, it is important to note that this law includes a provision for current owners of assault weapons to register their firearms with the state, ensuring that they can continue to legally possess them. This is a fair and reasonable solution that allows individuals to continue to exercise their Second Amendment rights while also taking steps to protect public safety.
I understand that this law may not be popular with everyone, but I believe it is a necessary step in addressing the ongoing issue of gun violence in our country. As your representative, I am committed to working towards solutions that keep our communities safe and I believe this law is a step in the right direction.
Thank you for your understanding and please do not hesitate to reach out if you have any further questions or concerns.
Sincerely,
Your thoughts?
- Kelly Cassidy - Friday, Jan 27, 23 @ 2:36 pm:
I’ve played with it to create fact sheets. I’ve fed a few bills into it and asked for bullet points and it’s surprisingly helpful.
- Rich Miller - Friday, Jan 27, 23 @ 2:36 pm:
===I’ve fed a few bills into it and asked for bullet points===
Fascinating. That could change a whole lot of things.
- Michelle Flaherty - Friday, Jan 27, 23 @ 2:41 pm:
Per the AI letter, people contacting a lawmaker are most likely upset with the assault weapons ban and specifically the require to register existing weapons with the state police.
It’s not a poorly crafted letter, but it reads like a form letter from a bureaucrat or computer and it likely serves to only further stoke the anger of the writer.
- DuPage Saint - Friday, Jan 27, 23 @ 2:44 pm:
I foresee another cut back amendment. Senate down to one Chat GBT and house down to two. No salaries no pensions no relative on payrolls. A win for Illinois
- cover - Friday, Jan 27, 23 @ 2:46 pm:
= the bot came in 36th out of 40 students in Con Law =
Did it place ahead of Tom Devore? /s
- G'Kar - Friday, Jan 27, 23 @ 2:48 pm:
I keep trying to access the ChatGPT website and I’m constantly unable connect. It must be popular.
- G'Kar - Friday, Jan 27, 23 @ 2:49 pm:
My last comment reads like it was written by a bot. I meant to write “I’m constantly unable to connect to the site.”
- levivotedforjudy - Friday, Jan 27, 23 @ 2:51 pm:
I’ve been doing my own little surveying. People who write essays have said they view it as a sort of editorial assistant. A lot of people I know who are more journalists (including me a Medill grad) view it as a tool for plagiarism. Is this really progress?
- Wensicia - Friday, Jan 27, 23 @ 2:52 pm:
The letter is pragmatic and concise, which is beyond the capability of the far right to understand.
- Jaded - Friday, Jan 27, 23 @ 2:58 pm:
I’ve seen how this story ends and it involves Neo, Morpheous, Trinity and/or John Connor trying to save the world…
- H-W - Friday, Jan 27, 23 @ 2:59 pm:
All of the ideas represented (as in, re-presented) in an AI essay are taken from existing sources. In the absence of sourcing, these are plagiarized essays.
That the user does not know the actual sources that are used to frame the essay can be problematic for people presenting these ideas as their own.
- OneMan - Friday, Jan 27, 23 @ 3:01 pm:
I shared on the earlier post its description of CapFax.
I asked it about a charitable organization I am involved with and it was on target until it added several countries to places the group is involved with.
That being said, when I asked it to describe symbolism in Moby Dick, it came up with an answer that would have made my HS English teacher happy.
- Dysfunction Junction - Friday, Jan 27, 23 @ 3:02 pm:
=The letter is pragmatic and concise, which is beyond the capability of the far right to understand.=
Not to mention NO CAPITAL LETTERS (banned punctuation). It’s not gonna fool anyone in the Eastern Bloc as having been from their legislator.
- Amalia - Friday, Jan 27, 23 @ 3:02 pm:
I’m most concerned with how to recognize that work comes from it. Don’t like it. Letter was meh.
- Sonny - Friday, Jan 27, 23 @ 3:06 pm:
Letter is vastly better than drafts of letters that I’ve had to rewrite or edit over the years
- very old soil - Friday, Jan 27, 23 @ 3:10 pm:
additional thoughts on the topic
https://jabberwocking.com/ai-is-still-not-here-but-its-getting-damn-close/
- Jerry - Friday, Jan 27, 23 @ 3:22 pm:
Does it order a take out pizza correctly? I’ll hangup and listen for my answer. (humor intended)
- clec dcn - Friday, Jan 27, 23 @ 3:22 pm:
Very impressive letter in many respects and far above what some might craft themselves. I have a friend who works in the AI world as a creator and designer. I believe he is paid handsomely for sure. It has potential for many things however not truly creative in the human sense. It also could depend on the slant of the creator or system. I wonder if I could give me a good Homily for Sunday Mass. Be interesting to find out.
- /s - Friday, Jan 27, 23 @ 3:23 pm:
Maybe I’m biased having written a number of these letters in my day but as an above comment stated, it reads like a form letter, with no tie-in to the district/community. This is less like having a world class chef in your kitchen and far more like having a PR firm based in NYC having their intern just out of college writing a letter for you, some representative in some town they’ve never heard of.
- Dysfunction Junction - Friday, Jan 27, 23 @ 3:24 pm:
Strange. I asked it to read me Doctor Memory and all it said was >SYSTAT UPTIME 9:01 unhappy READ MAKNAM”
In other words, still placed ahead of Tom DeVore.
- OneMan - Friday, Jan 27, 23 @ 3:27 pm:
Subject area AI will create challenges for the government sooner rather than later. One thing will be the requirements to retain and disclose training data for some systems, either for regulatory or litigation purposes.
If you feed training data that have biases present in them (obvious or not), those biases will tend to make their way into the AI and be harder to identify. Who is going to be liable for issues that become present in AI due to issues with the training data?
You are also looking at potential “ownership” issues down the road. Will vendors try to retain some ownership of results produced by their AIs for other entities? You can use the results for homework, but not a commercial product.
If some bot gives you bad investment advice, who is going to be liable? The person who used the bot, the folks who built the bot, the folks who provided training data? What if your data is used for training without your knowledge or permission? Can you still be liable for how it’s used? What if you give permission? What if you work someplace for five years, and they use what you have done as a work product to train a bot? What rights do you have?
Down the road, AI law is going to be a real thing and is going to be interesting.
- 47th Ward - Friday, Jan 27, 23 @ 3:30 pm:
For faculty worried about cheating, the absence of typos and use of excellent grammar should be a hint that the student used a chat bot.
You can tell my writing is original by all of the mistakes I make.
- Hannibal Lecter - Friday, Jan 27, 23 @ 3:35 pm:
=== I’m most concerned with how to recognize that work comes from it. Don’t like it. Letter was meh. ===
Unfortunately, many people would kill for “meh” nowadays. Gen Z should be concerned that they will not be needed because they will be replaced by robots.
- XonXoff - Friday, Jan 27, 23 @ 3:39 pm:
I ran a couple basic requests to generate python code a while back and was pretty impressed with the results.
- walker - Friday, Jan 27, 23 @ 3:55 pm:
Proft newspapers have an alternative source.
- don the legend - Friday, Jan 27, 23 @ 3:55 pm:
An experiment: A candidate for office should use it for every question asked throughout the campaign. It would be like Peter Sellers character, Chance, in Being There.
The answers would be so much better than most candidates provide. Think Darren Bailey.
- ZC - Friday, Jan 27, 23 @ 4:02 pm:
The most interesting thing I read today about ChatGPT, and how it “thinks,” was an exploration of why it still cannot generate academic citations and winds up making up stuff.
But it will get better, I’m sure.
https://twitter.com/dsmerdon/status/1618816703923912704
- JS Mill - Friday, Jan 27, 23 @ 4:14 pm:
Bailey and devore should consider using the bot for their communication. I too think the bot is bland and uninspiring, but unlike bailey and devore it is intelligible and makes sense most of the time.
- PF Flyer - Friday, Jan 27, 23 @ 4:22 pm:
It may have been a mediocre C+ but it passed the Bar exam, on the first try. It would result in a law degree.
- TheInvisibleMan - Friday, Jan 27, 23 @ 4:24 pm:
And this should be exactly how it needs to be integrated into learning.
Give students an assignment to knowingly use ChatGPT to generate your assignment, then have the students become their own teachers and check the output themselves for anything that is wrong and sign off on the final product. After handing in an assignment, it is then given to another student to do the same checking on the assignment of someone else, to sign off on the final product.
Which means they have to check every sentence themselves, and the same for questions someone else asks to be generated.
It integrates the new tool into education, with how it will most likely be used in their adult lives, while simultaneously showing it has weaknesses that can be found with additional checking and teamwork.
But, there’s no way that’s going to happen on a large scale in the US. Here, it will only be those who take it upon themselves to learn using this method, and it won’t take long before they leave the traditionally educated in the dust.
The old guy down the street who still talks about ’squaring the circle’ never did go on to be very good at math as an adult.
- Huh? - Friday, Jan 27, 23 @ 4:29 pm:
“If you feed training data that have biases present in them (obvious or not), those biases will tend to make their way into the AI”
A few years ago Amazon wrote an AI program in an effort to increase diversity in their hiring. The program kept hiring men because the AI program was trained by scanning the resumes of people that had been hired in the previous 10 years. After a few years, Amazon scrapped the system.
Just goes to show GIGO - garbage in, garbage out.
- Banish Misfortune - Friday, Jan 27, 23 @ 4:33 pm:
Constituent letter reads about like every letter I have ever received from a legislator.