Some neurons show high activation values matching administrative regions included in the input (e.g., New York state, Northern Ireland), while other neurons capture geographic patterns not explicitly mentioned in the prompt, such as southern vs northern Italy.
Method
Using mean pooling to convert token-wise activation embeddings into text activation embeddings. By combining these vectors with Longitude and Latitude data, we calculate Global/Local Moran's I indices to evaluate spatial autocorrelation of activation values. This measures how similarly embeddings are activated in geographically proximate locations using Global and Local Moran's I indices.