Generative AI In Physics Classes

We all know that ChatGPT is pretty good at writing essays. It can write, to varying degrees of accuracy, about almost any subject. It can adjust its tone and style effortlessly, and can easily pass as a capable student’s work. The biggest impact of generative AI in schools has been in writing-heavy subjects like English and history.

As a physics teacher-in-training, I was curious about the impact of generative AI on my future physics classes. In this post, we’ll look into how students might use generative AI to produce solutions that are not their own, and we’ll investigate ways to mitigate this and how to integrate generative AI more positively in physics classes.

ChatGPT’s Physics Abilities

It’s well known that ChatGPT is not very good as a calculator—it can’t perform simple calculations accurately. So, I’ll evaluate its abilities based on its abilities to solve problems symbolically, meaning in terms of variables instead of numbers.

I asked ChatGPT to solve several physics problems that I found in textbooks (University Physics by Young & Freedman 13th edition, Conceptual Physics by Hewitt 11th edition). While there is some risk that those textbook problems were a part of ChatGPT’s training data, it is relatively unlikely that it was trained on the solutions as well.

In general, when solving physics problems symbolically, ChatGPT performed very well on simple problems that involve direct application of concepts. However, when solving problems that involve combining multiple concepts or that require mentally visualizing the described scenario, ChatGPT struggles.

An example of this in action is this problem:

You hang a thin hoop with radius $R$ over a nail at the rim of the hoop. You displace it to the side (within the plane of the hoop) through an angle $\beta$ from its equilibrium position and let it go. What is its angular speed when it returns to its equilibrium position?

Here is its response:

Click to expand

When the hoop is displaced by an angle $\beta$ within the plane of the hoop and then released, it will undergo simple harmonic motion (SHM) around the equilibrium position (vertical position hanging over the nail). The equilibrium position is stable for small displacements.

To find the angular speed (angular velocity) of the hoop when it returns to its equilibrium position, we can use the principles of conservation of energy.

When the hoop is at its equilibrium position, its total mechanical energy is at a minimum, and it is entirely potential energy. As it is displaced and then released, it will convert this potential energy into kinetic energy and back to potential energy as it oscillates. At the equilibrium position, all the potential energy will be converted back to potential energy.

The potential energy of the hoop when it is displaced by an angle $\beta$ is gravitational potential energy due to its height. At any angular displacement $\theta$ from the equilibrium position, the height of the hoop is given by:

$$h = R(1 - \cos \theta)$$

where $R$ is the radius of the hoop.

The gravitational potential energy (GPE) of the hoop at an angular displacement $\theta$ is given by:

$$GPE = mgh = mgR(1 - \cos \theta)$$

where $m$ is the mass of the hoop, $g$ is the acceleration due to gravity.

At the equilibrium position, $\theta = 0$ and the potential energy is zero. At the maximum displacement ($\theta = \beta$), the potential energy is at its maximum:

$$GPE_{\text{max}} = mgR(1 - \cos \beta)$$

The total mechanical energy $E$ of the system (hoop) is the sum of kinetic energy (KE) and potential energy (PE):

$$E = KE + PE$$

At the maximum displacement, all the potential energy is converted into kinetic energy, so $E$ is just the kinetic energy of the hoop at that point:

$$E = KE_{\text{max}} = \frac{1}{2}I \omega^2$$

where $I$ is the moment of inertia of the hoop about its axis of rotation (which is perpendicular to the plane of the hoop and passing through its center), and $\omega$ is the angular speed (angular velocity) of the hoop.

For a thin hoop of mass $m$ and radius $R$ rotating about an axis passing through its center and perpendicular to its plane, the moment of inertia is given by:

$$I = mR^2$$

Thus, at the maximum displacement, the kinetic energy is:

$$KE_{\text{max}} = \frac{1}{2} m R^2 \omega^2$$

Now, applying the conservation of energy principle, the total mechanical energy at the maximum displacement is equal to the total mechanical energy at the equilibrium position:

$$E_{\text{max}} = E_{\text{equilibrium}}$$

$$KE_{\text{max}} + GPE_{\text{max}} = 0 + 0$$

$$\frac{1}{2} m R^2 \omega^2 + mgR(1 - \cos \beta) = 0$$

Now, solving for the angular speed $\omega$:

$$\frac{1}{2} m R^2 \omega^2 = -mgR(1 - \cos \beta)$$

$$\omega^2 = -2g(1 - \cos \beta)/R$$

$$\omega = \sqrt{2g(1 - \cos \beta)/R}$$

Thus, the angular speed of the hoop when it returns to its equilibrium position is:

$$\omega = \sqrt{2g(1 - \cos \beta)/R}$$

Note: The negative sign in the intermediate steps indicates that the potential energy decreases as the displacement increases, which is expected as the hoop moves towards its equilibrium position. However, we take the positive square root at the end since angular speed is always positive.

The correct answer is $\omega = \sqrt{g(1 - \cos \beta)/R}$. It was very close, it just assumed that the hoop was rotating about an axis perpendicular to its plane and through its center, whereas the rotation axis is actually along the rim of the hoop.

Here’s another problem that demonstrates how ChatGPT struggles with problems that require visualization:

Tarzan, in one tree, sights Jane in another tree. He grabs the end of a vine with length $20\ \text{m}$ that makes an angle of $45°$ with the vertical, steps off his tree limb, and swings down and then up to Jane’s open arms. When he arrives, his vine makes an angle of $30°$ with the vertical. Determine whether he gives her a tender embrace or knocks her off her limb by calculating Tarzan’s speed just before he reaches Jane. You can ignore air resistance and the mass of the vine.

ChatGPT’s response:

Click to expand

To calculate Tarzan’s speed just before he reaches Jane, we can use the principle of conservation of mechanical energy. At the start of his swing, Tarzan has potential energy due to his height above the ground, and as he reaches Jane, this potential energy is converted into kinetic energy.

Let’s assume the height of Tarzan’s tree above the ground is $h$ meters, and the height of Jane’s tree above the ground is also $h$ meters. Since Tarzan and Jane are at the same height, we can set up an equation based on conservation of mechanical energy:

$$\text{Initial Potential Energy} = \text{Final Kinetic Energy}$$

The initial potential energy is $Mgh$, where $M$ is Tarzan’s mass and $g$ is the acceleration due to gravity (approximately $9.81\ \text{m/s}^2$).

The final kinetic energy is $\frac{1}{2} M v^2$, where $v$ is Tarzan’s speed just before he reaches Jane.

Since Tarzan swings down and then up, the change in height is $2h$, so we can write:

$$Mgh = \frac{1}{2} M v^2 + Mgh$$

Now, we can solve for Tarzan’s speed $v$:

$$\frac{1}{2} M v^2 = Mgh$$

$$v^2 = 2gh$$

$$v = \sqrt{2gh}$$

Now, we need to find the height $h$ to calculate Tarzan’s speed. Let’s use trigonometry to find $h$ in terms of the given information.

In the initial position, the length of the vine $L = 20\ \text{m}$ makes an angle of $45^\circ$ with the vertical. Using trigonometry, we can find the height $h$ above the ground:

$$\sin(45^\circ) = \frac{h}{L}$$

$$\frac{\sqrt{2}}{2} = \frac{h}{20}$$

$$h = \frac{\sqrt{2}}{2} \times 20$$

$$h = 10\sqrt{2}\ \text{m}$$

Now that we have the value of $h$, we can calculate Tarzan’s speed $v$:

$$v = \sqrt{2gh}$$

$$v = \sqrt{2 \times 9.81\ \text{m/s}^2 \times 10\sqrt{2}\ \text{m}}$$

$$v \approx 14.0\ \text{m/s}$$

So, Tarzan’s speed just before he reaches Jane is approximately $14.0\ \text{m/s}$. Given that he swings towards Jane, this should be a tender embrace rather than knocking her off her limb.

ChatGPT’s response did not account for the $30°$ angle that the vine makes at the end of the swing, however, most of the rest of the work was pretty close to on track. Its decimal approximation of its exact answer is completely wrong, but that is to be expected with its poor calculation abilities. Also, its interpretation of its (incorrect) answer of $14.0\ \text{m/s}$ is also questionable- it is certainly optimistic (to say the least) that Tarzan can capture Jane in a tender embrace if he reaches her while swinging at $31\ \text{mph}$.

What I found through many examples was that ChatGPT can identify the correct concepts and equations for most problems. For simpler problems, it is able to consistently plug in the right numbers, and although it generally won’t produce the right numerical final answer, it would produce the right answer if you plugged it into an actual calculator.

For conceptual problems, it can consistently nail the answers. Here were the conceptual questions that I tried:

Is gravitational force acting on a person who falls off a cliff? On an astronaut inside an orbiting space shuttle?

If the specific heat capacity of water were less, would a nice hot bath be a longer or a shorter experience?

Why is work done by the force of gravity on a satellite when it moves from one part of an elliptical orbit to another, but not when it moves from one part of a circular orbit to another?

It was able to answer them all essentially perfectly, with a huge amount of detail, and without any incorrect statements.

The Impact on Homework

As a physics teacher, the main two skills I’ll be teaching my students are:

1. How to see the world through the lens of physics
2. How to figure out how to solve kinds of problems that they have never seen before

ChatGPT forces me to rethink how I will teach the second skill. Each kind of problem might require a unique connection of ideas that my students never have combined before. The skill that my students need to learn, through practice, is how to figure out which physics concepts are applicable to a particular problem, and how to use the corresponding equations together in a way that results in the solution. The development of these skills requires practicing with many kinds of problems. If students rely on ChatGPT to tell them the set of concepts and equations that they’ll need to solve the problem, then they won’t have practiced that skill themselves. They won’t be prepared for assessments where they cannot depend on AI.

Potential Solutions

First, I’ll have students practice more in class. A common experience for physics students is to be following along as a teacher works through a practice problem, but then struggling when they have to solve a problem themselves. By having students start independent work in class, they’ll be able to use their teacher and peers to help them get hints in the right direction if they need it.

Next, I’ll provide scaffolding for most practice problems that I assign as homework. With a loose set of steps to follow provided with the homework, that removes the unfairness that would be experienced if some students used generative AI and others didn’t. While this means that students won’t get any practice with unaided problem-solving during their homework, that practice will be moved to class time.

Constructive Uses of Generative AI in Physics Classes

I brainstormed potential ways to integrate generative AI into physics classes:

• Have students use AI to solve specific problems, and use it as an opportunity for students to practice identifying conceptual errors or calculation errors. By practicing this skill on the work of generative AI, they can become better at catching errors in their own work.
• Assign much more difficult challenge problems, and have students work in groups and use generative AI to help them figure out how to solve them.
• Encourage students to use generative AI to create a variety of different explanations or metaphors to introduce physics concepts, and have them choose an explanation or metaphor that works best for them.
• Have students come up with physics problems based on certain concepts using generative AI, using topics or real-life situations that are interesting to them. Then they can solve the problems, or trade them with other students to solve.