pyc • Lines: 189�
��j' �
� � d dl Zd dlmc mZ d dlZd dlmZ d dl m
Z
d dl m Z
d dlZd dlZ ej" ej$ j' � rdnd� Zg d�Zdj+ e� j- � Z e ee� � Z ee� D � �ci c] \ } }|| ��
c}} Z e e� Z g Z eD ]Z Z e j- � Z! e"d e e!� dz
� D ]3 Z#e e!e#dz
e e!e#dz gZ$e e!e# Z%e jM e$e%f� �5 �\ ejN e D �cg c] }|d �� c}ejP �� jS e� Z* ejN e D �cg c] }|d �� c}ejP �� jS e� Z+ G d� d ejX � Z- e-e d
� � jS e� Z. ej^ � Z0 e
jb e.je � d �
� Z3d� Z4 e
jj � e
jl d�� \ Z7Z8g d�Z9g d�Z: e"d� D �]� Z;e.jy � e3j{ � e0 e.e*� e+� Z>e>j � e3j� � e;dz d k( s�\e8j� � ej� � 5 e.j� j� ZEeEeEj� dd�� dz z j� � j � ZH e4eH� ZIddd� e9D ][ ZJeJe v s�e eJ ZKeIeK ZLeJdv rdneJdv rd nd ZMe8j� eLd eLd eMd d d �!� e8j� eJeLd eLd fd"d#d$d%�&� �] e:D ]I \ ZPZQZMZRePe v s�
eQe v s�eIe eP eIe eQ cZSZTe8j� eSd eTd geSd eTd geMeRd'd(�)� �K e8j� d*e;� d+e>j� � d,��� e8j� dd-d.�/� e8j� d0� e
j� d1� ��� e
j� � e
j� � yc c}} w c c}w c c}w # 1 sw Y ��KxY w)2� N�cuda�cpu)!z man is malezwoman is femalez king is malezqueen is femalezpalash is maleztotan is femalez boy is malezgirl is femalezthe man is the kingzthe woman is the queenzpalash is the kingztotan is the queenzpalash is a manztotan is a womanzpalash is husbandz
totan is wifezking loves queenzqueen loves kingzpalash loves totanztotan loves palashzprapti is girlztripti is girlzprapti is princessztripti is princesszprapti is daughterztripti is daughterz prapti and tripti are sisterszpalash father praptizpalash father triptiztotan mother praptiztotan mother triptizking father princesszqueen mother princess� � )�dtypec �& � � e Zd Zd� fd� Zd� Z� xZS )�UltimateEmbeddingModelc � �� t t | � � t j ||� | _ t j ||� | _ y )N)�superr �__init__�nn� Embedding� embedding�Linear�fc_out)�self�
vocab_size�d_model� __class__s ��?/home/palash/git/iron_learn/python_scripts/test_vector_space.pyr z UltimateEmbeddingModel.__init__< s5 �� �
�$�d�4�6�� � �j�'�:����i�i���4�� � c � � | j |� }||j dd�� dz z }| j |j d�� � S )N�����T��dim�keepdim�:�0�yE>r )r )r �normr �mean)r �x�embeddeds r �forwardz UltimateEmbeddingModel.forwardA sF � ��>�>�!�$���x�}�}��T�}� B�T� I�J���{�{�8�=�=�Q�=�/�0�0r )� )�__name__�
__module__� __qualname__r r" �
__classcell__)r s @r r r ; s
�� �5�
1r r r# )r r g{�G�zt?)�lrc �. � | t j | d�� z
}t j |j � }t j j |� \ }}t j |� d d d� }t j ||d d �|d d f � S )Nr )�axisr � )�npr �cov�T�linalg�eigh�argsort�dot)�X�X_meanr- � eigenvalues� eigenvectors�idxs r � apply_pcar8 K sw � �
������#�
#�F�
�&�&����
�C� "� � ���s� 3� �K� �
�*�*�[�
!�$�B�$�
'�C�
�6�6�&�,�q�#�b�q�'�z�2� 3�3r )� �
)�figsize)
�man�woman�king�queen�palash�totan�prapti�tripti�princess�girl) )r< r> �blue�--)r= r? �redrG )r@ r> �purple�-)rA r? rI rJ )r@ r< rF �:)rA r= rH rK )rB rD �orangerJ )rC rD rL rJ )rE rD �magentarG )r@ rB �greenrK )rA rC rN rK i� �d r Tr r )r< r> r@ rF )r= r? rA rH rN � �k� )�c�s�
edgecolors�zorder)� rW z
offset points� �bold)�xytext�
textcoords�fontsize�
fontweightg �?r+ )�color� linestyle�alpha� linewidthz2Ultimate Family Semantic Space (PCA 4D->2D)
Epoch z | Loss: z.4frG g�������?)r_ r` g�������?g{�G�z�?)]�builtins� @py_builtins�_pytest.assertion.rewrite� assertion�rewrite�
@pytest_ar�torch�torch.nnr
� torch.optim�optim�matplotlib.pyplot�pyplot�plt�math�numpyr, �devicer � is_available�corpus�join�split�words�list�set�vocab� enumerate�
word_to_ix�lenr �data�sentence�tokens�range�i�context�target�append�tensor�long�to�x_train�y_train�Moduler �model�CrossEntropyLoss� criterion�Adam�
parameters� optimizerr8 �ion�subplots�fig�ax� target_words� conn_groups�epoch�train� zero_grad�loss�backward�step�clear�no_gradr �weight�
raw_embedsr r � norm_embeds�
embeddings_2d�wordr7 �vecr^ �scatter�annotate�w1�w2�style�v1�v2�plot� set_title�item�grid�margins�pause�ioff�show)r� �w�pairs 000r �<module>r� sS �� � � � � � � � � �� � ��
�
� 7� 7� 9�f�u� E��
��< ����� � � �� �S��Z��� (�� /�
0�t�q�!�a��d�
0�
�
��Z�
�
��� '�H�
�^�^�
�F�
�1�c�&�k�A�o�
&� '�� �f�Q�q�S�k�*�J�v�a��c�{�,C�D���F�1�I�&�� � � �W�f�%�&�'�'� �%�,�,�D�1�D��Q��1����
D�
G�
G��
O��
�%�,�,�D�1�D��Q��1����
D�
G�
G��
O��
1�R�Y�Y�
1� �*�a�@�C�C�F�K�� �B� � � !� � �E�J�J�u�'�'�)�e� 4� �4� ���� �
�#�,�,�x�
(���R�k� �� � �6�]� �E� �K�K�M�
���� �U�7�^�W� -�D��M�M�O�
�N�N�� �s�{�a��
���
�
�U�]�]�_� 3� � � �/�/�J�%�� � �R�� �)N�QU�)U�V�[�[�]�c�c�e�K� %�k� 2�M� 3�
!� �D��z�!� ��&��#�C�(��"&�*C�"C��RV�Zu�Ru�� |C���
�
�3�q�6�3�q�6�U�c�c�RS�
�T�� � �D�3�q�6�3�q�6�"2�6�o�hj�w}� �~�
� %0� n� �B��E�5��Z� �B�*�$4�&�z�"�~�6�
�j�QS�n�8U���B�����A���1����A���1���e�u�\_�kl��m� n�
� � �J�5�'�QZ�[_�[d�[d�[f�gj�Zk�l�m�
�����C��0�
�
�
�3���� � �$��? �B ����
� ����
��q 1�� 2��1�f 3� 3�s �
P1� P7� P<�$AQ�Q