Table Comparing Scores, Ratings, and Percentages with Real Measures

(Documentation to be posted tomorrow.)

Characteristics Raw Scores and/or Percentages Rasch Measurement

Quantitative hypothesis

Neither formulated nor tested

Formulated and tested

Criteria for falsifying quantitative hypothesis None Additivity, conjoint transitivity, parameter separation, unidimensionality, invariance, statistical sufficiency, monotonicity, homogeneity, infinite divisibility, etc.
Relation to sample distribution Dependent Independent

Paradigm

Descriptive statistics

Prescriptive measurement

Model-data relation

Models describe data, models fit to data, model with best statistics chosen

Models prescribe data quality needed for objective inference, data fit to models, GIGO principle

Relation to structure of natural laws

None

Identical

Statistical tests of quantitative hypothesis None Information-weighted and outlier-sensitive model fit, Principal Components Analysis, many other fit statistics available
Reliability coefficients Cronbach’s alpha, KR-20, etc. Cronbach’s alpha, KR-20, etc. and Separation, Strata
Reliability error source Unexplained portion of variance Mean square of individual error estimates
Range of measurement Arbitrary, from minimum to maximum score Nonarbitrary, infinite
Unit status Ordinal, nonlinear Interval, linear
Unit status assumed in statistical comparisons Interval, linear Interval, linear
Proofs of unit status Correlational Axiomatic; reproduced physical metrics; graphical plots; independent cross-sample recalibrations; etc.
Error theory for individual scores/measures None Derived from sampling theory
Architecture (capacity to add/delete items) Closed Open
Supports adaptive administration and mass customization No (changes to items change meaning of scores) Yes (changes to items do not change meaning of measure)
Supports traceability to metrological reference standard No Yes
Domains scored Either persons or items but rarely both All facets in model (persons, items, rating categories, judges, tasks, etc.)
Comparability of domains scored Would be incomparable if scored Comparable; each interpreted in terms of the other
Unscored domain characteristics Assumed all same score or random (though probably not)
No unscored domain
Relation with other measures of same construct Incommensurable Commensurable and equatable
Construct definition None Consistency, meaningfulness, interpretability, and predictability of calibration/measure hierarchies
Focus of interpretation Mean scores or percentages relative to demographics or experimental groups Measures relative to calibrations and vice versa; measures relative to demographics or experimental groups
Relation to qualitative methods Stark difference in philosophical commitments Rooted in same philosophical commitments
Quality of research dialogue Researchers’ expertise elevated relative to research subjects Research subjects voice individual and collective perspectives on coherence of construct as defined by researchers’ questions
Source of narrative theme Researcher Object of unfolding dialogue

Creative Commons License
LivingCapitalMetrics Blog by William P. Fisher, Jr., Ph.D. is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States License.
Based on a work at livingcapitalmetrics.wordpress.com.
Permissions beyond the scope of this license may be available at http://www.livingcapitalmetrics.com.

Advertisements

Tags: , , ,

2 Responses to “Table Comparing Scores, Ratings, and Percentages with Real Measures”

  1. Matt Barney Says:

    Brilliant – would love permission to share with people I coach / teach about Rasch measurement

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: