Attendnace

LLM

• Large Language Model
• Language-based mimic
• Capable of abstraction
  • Recognize and apply learned patterns
• Incapable of “hard logic”
• Garbage-in-garbage-out

Intrinsic limitations of LLM

• Linguistic-only.
• Do we think in words?
• Is creativity linguistic-only?

Q-Star

• New development from OpenAI
• The exact nature is not known
• Speculations:
  • A-Star, combined with
  • Q learning
• Should resolve some “hard logic” limitations

The point is…

• Development is fast
• Lots of active research both in academia and corporate world
• Moore’s Law is still alive, but aged
• AI will be more capable and accessible, period

Here we are

• What do our students need?
• How does AI integrate into what we do?
• What is our (human) value in education?

Challenges

• Students using AI to do their homework
• Can AI replace us?
• Can we use AI to enhance teaching?
• What matters in the end?

What matters in the end?

• Are our students hirable?
• Are our students responsible members of society?
• Are our students “educated”?

Will you hire our students?

• What do you expect them to accomplish at work?
• Is AI a useful tool you expect employees to use?
  • If so, which part of a job still needs human intervention?
• How is AI utilized in the job?

What does Sir Christopher Pissarides have to say?

• IT jobs sow their “own seeds of self-destruction.”
• Some skills are less “replaceable by AI”:
  • Managerial
  • Creative
  • Empathetic
  • Really?

I’m sorry, Sir Pissarides

• Yeah, not really!
• AI can:
  • distinguish human facial expressions better than an average person,
  • generate images, music, writing, etc. in specific styles,
  • “understand” rules, apply rules in situations, and do so while sounding empathetic.

Let’s try this again

• A LLM relies on massive training samples.
• If something is not well documented/discussed, a LLM cannot learn it.
• A LLM has no sensory input/association.
• A LLM does not understand hard logic.

What can (some) humans (still) do better?

• Critical thinking (application of hard logic).
• On-the-spot problem-solving (no prior examples, plus the application of hard logic).
• Abstraction, theorization. The recognition of systematic or cross-domain conceptual (as opposed to linguistic) pattern.
• Learning using few examples, or just the explanation of rules.

Curricula and SLOs

• What do we focus on?
• What to teach?
• How to assess?
• Bloom’s Taxonomy (from 1956!).

Bloom’s Knowledge

• still important, where to find information
• authority of the source of information
• reliability of the source of information
• not so much the memorization of information

Bloom’s Comprehension

• the relationship of pieces of knowledge
• if definitions, terms, etc. are nodes, comprehension is the mesh connecting the nodes

Bloom’s Application

• Was Benjamin Bloom right to put application between “comprehension” and “analysis”?
• Is “application” the same as “problem-solving”?

Bloom’s Analysis

• Given a problem/scenario and solution description, break it up and relate to concepts, definitions.
• Applicable to a problem/scenario description.
• Also, applicable to the qualification of a solution.
• Mapping a problem/scenario and solution description as instances onto the mesh of knowledge and comprehension
• The first step of problem-solving.

Bloom’s Synthesis

• Given the results of analysis, the derivation of actions/steps to achieve a target scenario.
• Think of the problem description as an initial state as a super-imposed mesh over the knowledge-comprehension mesh.
• Each action morphs this super-imposed mesh.
• How to morph the super-imposed mesh to match the final state, a solution as a super-imposed mesh?
• There may be multiple ways to achieve a target scenario.

Bloom’s Evaluation

• Compare and contrast the various ways generated in synthesis.
• Apply criteria and requirements that are applicable to compare alternatives.
• Also, a way to evaluate scenarios.

Designing assessment activities

• To deter/detect the use of AI tools

Contextualize questions

• Based on what we discuss in class…
• After watching this video on YouTube,…
• Describe your personal experience and relate it to…

In-person assessment

• In-person
• On-paper
• No electronics!

Google Doc

• Use edit tracking
• A human edit history should be obvious
• Ask Tak for ideas to use a script to measure metrics!
• Will need editor access
  • Generate instructor owned document and share with students