What a number leaves out

Consider a university deciding how students should reach its support service. This is an invented case. There are no survey findings hiding behind the example, and the numbers that follow describe hypothetical designs.

The first design offers quick appointments. To make that possible, the service assigns students to the next available adviser, asks them to select a problem from a short menu, and limits changes after booking. The second design gives students more say over whom they meet, what they disclose, and how a meeting proceeds. It takes longer to arrange.

Someone must choose. The budget will not fund both designs indefinitely. A committee asks for a number that will settle the question.

This sounds like the moment for analysis to begin. It is already well underway. The appointment system has decided when waiting starts. The booking menu has decided which problems exist. The database has decided that a student who never completed the form is different from a student who waited for an appointment. A column of apparently ordinary observations contains the accumulated consequences of those decisions.

Defining access

Suppose the service records the time between a submitted request and an appointment. That is a useful duration. It leaves out the student who opened the form repeatedly before submitting it, the student who needed help understanding it, and the student who abandoned it because none of its categories fitted.

These are possible failures to investigate, not findings about an actual service. Their importance lies in how they change the object of measurement. If the question is how efficiently the office processes completed requests, the recorded duration may be appropriate. If the question is how easily students obtain help, the same duration is incomplete.

More accurate timestamps will not recover the missing attempts. A larger sample of completed requests will not make those attempts appear. The measurement problem concerns who and what can enter the record.

Now consider participant control. A booking system might record the number of options presented. Five unusable options do not necessarily offer more control than one suitable option that a student can freely decline. An interface might permit rescheduling while making the consequences of doing so unclear. A form might offer an open text box after every consequential decision has already been made.

For this case, we can begin with observable features: whether an adviser can be changed, whether a question can be skipped, whether an appointment can be declined without losing access, whether the consequences are explained. Each feature is open to inspection. Their relationship to a broader idea of participant control requires argument and evidence.

Calling the resulting total an autonomy score would not supply that evidence. Calling it a dignity score would create an even larger obligation. Human worth is not the quantity under construction here. The proposed object is a service and the opportunities it affords.

Averages on an ordinal scale

Imagine that a separate exercise asks reviewers to place service encounters into four ordered categories. Category 4 is preferred to category 3, which is preferred to category 2, which is preferred to category 1. Nothing establishes equal distances between them.

Two encounters under design A receive categories 1 and 4. Two under design B receive categories 2 and 3. The average of each pair is 2.5. A spreadsheet announces a tie.

It has quietly treated the distance between categories 1 and 2 as equal to the distance between categories 3 and 4.

Keep every encounter in precisely the same order, but give the categories curved numerical spacing. Compress the upper end, and B has the larger mean. Stretch the upper end, and A does. Both recodings can preserve the bottom and top anchors. Nobody has changed a judgment about which encounter outranks another. The average has changed its verdict because the coding has supplied information the original judgments did not contain.

The field guide works through the arithmetic. The lesson is not that averages are forbidden. An average makes a claim about the scale on which it is calculated. That claim needs support.

Some conclusions survive this recoding. A contains both the lowest-ranked and the highest-ranked encounter. B contains the two middle encounters. Those are facts about order. Whether a design should be rewarded for its best encounter or penalized for its worst is a further question.

Weights and uncertainty

Return to the service designs. For a classroom investigation, assume that we have constructed two comparable, anchored rubrics running from 0 to 100: one for access and one for participant control. Higher is better within each rubric. Treating the intervals as meaningful is an explicit assumption of this exercise. These are invented scenario scales, with no claim to validate a real instrument.

Available information places A’s access between 75 and 85 and its control between 25 and 45. For B, access lies between 45 and 65 and control between 65 and 85. A has the advantage on access. B has the advantage on control.

Give the dimensions equal weight and use the middle of each interval. A receives 57.5; B receives 65. The meeting could end here, with a recommendation to adopt B.

But the middle of a range is not automatically its best estimate. These ranges describe compatible possibilities. They are not probability distributions, and they are not sampling confidence intervals. Using their centers is a convenient illustration, not an evidential entitlement.

Under equal weights, the compatible gap between A and B extends from −25 to 10. Some allowed profiles favor B. Others favor A. The apparent certainty of the recommendation came from discarding the possibilities on either side of the centers.

There is another choice in the calculation. Equal weights express a particular trade-off on these particular scales. They do not arise naturally from having two columns.

Put nine-tenths of the weight on access, and every compatible profile favors A: its advantage ranges from 3 to 34. Put one-tenth on access, and every compatible profile favors B: A’s gap ranges from −53 to −14.

Better evidence about the designs could narrow the profile ranges. It would not, by itself, tell the committee how much access should be exchanged for participant control. A study of preferences might inform that discussion. Choosing whose preferences count, and under what constraints, remains part of the decision.

The service measurements and the weights raise separate questions. Additional observations may narrow the score ranges while leaving disagreement over the weights unresolved.

Revising the measurement procedure

One possibility is to inspect the features responsible for the wide control intervals. Another is to examine the assumed boundaries of the instrument itself.

Imagine planning interviews with students who completed a booking, students who began but withdrew, and people who helped someone else navigate the process. The aim would include discovering whether the current categories miss an important kind of constraint. Can a student decline a suggested option without fearing the loss of support? Does an apparently convenient appointment require them to miss paid work? Does the process become easier for those who already know the institution’s vocabulary?

An account of a failed attempt could force a revision to the definition of access. An account of an unwanted disclosure could reveal that the proposed control rubric measures the wrong choices. Neither contribution needs to be converted into a sentiment score before it can change the analysis.

Such accounts also need critical handling. A vivid example does not establish prevalence. An interview sample can exclude the very people a study hopes to understand. An interpretation should be answerable to the material it interprets and open to competing readings.

Quantitative and qualitative work can meet at this point: each can expose an assumption the other has allowed to pass unnoticed. The result may be a new variable, a revised sampling frame, a narrower claim, or the abandonment of a proposed score.

Language models might help locate passages for human review in a suitably governed study. Their labels would then become another observation channel to examine. Performance would need checking against held-out human judgments, including disagreements and failures in the contexts where the labels will be used. Fluent explanations would not establish validity. A model can make a mistaken classification unusually easy to read.

No personal accounts or automated psychological assessments are collected by the browser experiments accompanying this essay. Their purpose is to make the structure of these choices inspectable with synthetic inputs.

Incomplete rankings and minimum standards

If the committee cannot justify a common trade-off, the access and control profiles can remain separate. They still convey information. If one design were at least as good on both dimensions and better on one, an agreed preference for higher values would support a dominance comparison. In this example, the dimensions pull in opposite directions.

That is incomparability under the stated information and commitments. It is different from equality. A tie would say the alternatives have the same value under a specified rule. Incomparability says the rule and evidence do not support an ordering.

The committee might also set a requirement that cannot be compensated by faster access. Perhaps an acceptable service must permit a particular form of refusal. That requirement can be represented as a constraint: remove designs that fail it before comparing the remaining options. Doing so makes the commitment explicit. It also reveals a practical possibility that a single score obscures: neither current design may be acceptable.

An institution still has to act. It can use a provisional rule, revise a design, gather targeted evidence, or explain a decision made under unresolved disagreement. Analysis can clarify the consequences of these choices without pretending that the available information has settled them.

Further study

The support-service problem crosses several fields. Econometrics asks what the observations identify. Measurement theory asks which relationships a scale represents and which statements survive legitimate recoding. Qualitative inquiry can uncover distinctions the observation system cannot yet express. Decision theory makes uncertainty, preferences, and constraints visible in the choice itself.

The connections are established intellectual resources. The proposed teaching project is to make them part of one continuous investigation, and to test whether that continuity helps people reason better. The technical companion separates the underlying results from the research questions still open.

A first-year student can discover the problem by watching an average reverse. An undergraduate can derive the set of compatible score gaps. A doctoral researcher can ask whether those bounds are sharp, whether the model survives outside its original setting, and which new observation would be worth collecting. They can be working on the same question without pretending to have the same preparation.

The examples distinguish three sources of uncertainty: incomplete observation, unsupported assumptions about a scale, and unresolved value judgments. Each requires different evidence or deliberation.