← Posts

Thread · 2 posts

MetaRepICL

Testing whether kernel regression describes a transformer's in-context computation. The thread moves from construction sketches to trained-model comparisons and coordinate controls that overturned the original CG interpretation.

  1. Started
  2. Last updated

Posts follow their reading order. Each shows its publication date and latest revision.

Reading path

The double ring marks the latest post.

  1. Post 01

    MetaRepICL: Is In-Context Learning Kernel Regression?

    MetaRepICL studies how attention can carry out kernel regression, then tests whether that description predicts the behavior of trained transformers.

    Published
    Updated

    Test the proposed computation

    The construction motivates comparisons with trained transformers. Coordinate permutations then test whether agreement with a solver's prediction trajectory identifies the computation.

  2. Post 02Latest post

    Trained Models, Probing, and Silent Failures

    Comparing trained regression models with iterative solvers, then testing whether the agreement survives a permutation of the input coordinates.

    Published
    Updated