Skip to content

shimo-lab/Universal-Geometry-with-ICA

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

14 Commits
 
 
 
 
 
 
 
 

Repository files navigation

Universal-Geometry-with-ICA

Discovering Universal Geometry in Embeddings with ICA
Hiroaki Yamagiwa*, Momose Oyama*, Hidetoshi Shimodaira
EMNLP 2023

English word embeddings

Heatmap of ICA-transformed word embeddings

heatmap

Cross-lingual embeddings

Heatmaps of ICA-transformed word embeddings

cross-lingual heatmap

Spiky shape of embedding distributions

ica shape

Scatter plots of ICA-transformed word embeddings

English Spanish
ica en ica es
Russian Arabic Hindi Chinese Japanese
ica ru ica ar ica hi ica zh ica ja

Code and Data

  • The code for English embeddings is currently being prepared.
  • For cross-lingual embeddings, dynamic embeddings, and image model embeddings, please refer to the universal directory.

Citation

If you find our code or data useful in your research, please cite our paper:

@inproceedings{DBLP:conf/emnlp/YamagiwaOS23,
  author       = {Hiroaki Yamagiwa and
                  Momose Oyama and
                  Hidetoshi Shimodaira},
  editor       = {Houda Bouamor and
                  Juan Pino and
                  Kalika Bali},
  title        = {Discovering Universal Geometry in Embeddings with {ICA}},
  booktitle    = {Proceedings of the 2023 Conference on Empirical Methods in Natural
                  Language Processing, {EMNLP} 2023, Singapore, December 6-10, 2023},
  pages        = {4647--4675},
  publisher    = {Association for Computational Linguistics},
  year         = {2023},
  url          = {https://aclanthology.org/2023.emnlp-main.283},
  timestamp    = {Wed, 13 Dec 2023 17:20:20  0100},
  biburl       = {https://dblp.org/rec/conf/emnlp/YamagiwaOS23.bib},
  bibsource    = {dblp computer science bibliography, https://dblp.org}
}