pyc • Lines: 177�
��}j�
� �8 � d dl Z d dlZd dlZd dlZd dlZd� Zd� Zy)� Nc � � t j dd| t j �� } t j dd| t j �� } t j dd| � } t j dd| � j � } | S )Nz<style[^>]*>.*?</style>� )�flagsz<script[^>]*>.*?</script>z<[^>]+>� z\s+)�re�sub�DOTALL�strip��texts �-/home/palash/git/local-ai/server/read_file.py�
strip_htmlr sf � �
�6�6�,�b�$�b�i�i� H�D�
�6�6�.��D�� � � J�D�
�6�6�*�c�4� (�D�
�6�6�&�#�t� $� *� *� ,�D� �K� c �z � t j j | � d j � }t | d� 5 }|j � }d d d � |dk( rl dd l}|j d�� }g }|D ]! }|j |j � � �# |j � dj |� }t |� S |dk( rCddlm} | t/ j0 � � }dj d� |j2 D � � S |dk( r�t j dd� � 5 } | j � | j }
d d d � t# j$ d
gd d d
�� } | j4 dk( r!| j& t j( |
� S t# j$ d|
gd d d
�� } | j& t j( |
� S |dv r�ddlm }
|
t/ j0 � d d �� }g }|j: D ]@ }|j= d �� D ]) }|j dj d� |D � � � �+ �B |j � dj |� S t? d� |jA d � S # 1 sw Y ��ExY w# t $ r� t j dd� � 5 } | j � | j }
d d d � n # 1 sw Y nxY w t# j$ d
d gd d d
�� } | j& t j( |
� cY S # t j(
� w xY ww xY w# 1 sw Y ��xY w# t j(
� w xY w# tB $ r; t? d � jA d � cY S # tD $ r t? d � Y Y y w xY ww xY w)!N� �rbz.pdfr �pdf)�stream�filetype�
F)�suffix�delete� pdftotext�-T� )�capture_outputr �timeoutz.docx)�Documentc 3 �4 K � | ] }|j �� � y �w)Nr )�.0�ps r
� <genexpr>z!read_file_text.<locals>.<genexpr>, s � �� �8�A����8�s �z.doc�catdoc�antiword)z.xlsz.xlsx)�
load_workbook)� read_only� data_only)� values_only� c 3 �: K � | ] }|� t |� nd�� � y �w)Nr )�str)r �cs r
r" z!read_file_text.<locals>.<genexpr>C s � �� �%S�!��
�c�!�f�2�&E�%S�s �zReading code file(s)zutf-8z Trying Latinzlatin-1� Exceptionr )#�os�path�splitext�lower�open�read�fitz�append�get_text�close�joinr � ImportError�tempfile�NamedTemporaryFile�write�name�
subprocess�run�stdout�unlink�docxr �io�BytesIO�
paragraphs�
returncode�openpyxlr% �
worksheets� iter_rows�print�decode�UnicodeDecodeErrorr- )� file_path�ext�f�rawr4 �doc�lines�pager �tmpf�tmp�rr r% �wb�rows�sheet�rows r
�read_file_textr[ s5 � �
�'�'�
�
�9�
%�a�
(�
.�
.�
0�C�
�i�� � �!��f�f�h���
�f�}� � ��)�)�3��)�7�C��E��
.��� � �T�]�]�_�-�
.� �I�I�K��9�9�U�#�D� �d�#� #�
�� �!��r�z�z�#��'���y�y�8�� � �8�8�8� �� �
�
(�
(��u�
E� �� �J�J�s�O��)�)�C� � �����3���4���A� �|�|�q� ��x�x�
�I�I�c�N� ����S�!�$�T�2��A� �8�8� �I�I�c�N� �!� !�*�
�2�:�:�c�?�d�d�
K�����]�]� U�E� ���4��8�
U��� � �D�I�I�%S�s�%S� S�T�
U� U� ���
��y�y��� � � �(�#� .��:�:�g�&� &�q� �� �
��,�,�F�5�I�
�T��
�
�3���i�i��
�
�
��
� �N�N� �#�s�+�D�t�UW��� �x�x�� � �#���� � �#���
�� � ��
�I�I�c�N�� "� �
��n�%��z�z�)�,�,�� �
��k�"��
�� �s� �J� A*J �+ M�5M � &M �4 M6 �J� M �> K%� M �%K. �*M �2'L1�M �1M�M �M� M3�6 N:� N �N:� N6�2N:�5N6�6N:)rC r. r r> r: r r[ � r r
�<module>r] s �� � � � � ��Br