| Server IP : 121.121.20.254 / Your IP : 216.73.216.165 Web Server : Microsoft-IIS/10.0 System : Windows NT WEB-SERVER 10.0 build 20348 (Windows Server 2022) AMD64 User : IUSR ( 0) PHP Version : 8.3.28 Disable Function : NONE MySQL : ON | cURL : ON | WGET : OFF | Perl : OFF | Python : OFF | Sudo : OFF | Pkexec : OFF Directory : /Python315/Lib/urllib/__pycache__/ |
Upload File : |
R
>lqj�2 c � � Q s ] PHs] PHs] PHs] PHs] PHsQ,s\O QQ2 s Q Q2 s
Q Q2 s Q Q2 sQ s
Q
sQ sQ sQ
sP! )z�robotparser.py
Copyright (C) 2000 Bastian Kleineidam
You can choose between two licenses when using this package:
1) GNU GPLv2
2) PSF license for Python 2.2
The robots.txt Exclusion Protocol is implemented as specified in
RFC 9309
�RobotFileParser�RequestRatezrequests secondsc �x ` � \ sQ s]s n QsQQ ksQ sQ sQ sQ s Q s
Q sQ sQ
s
Q sQ sQ
sQ sQsU sP! )r zjThis class provides a set of methods to read, parse and answer
questions about a single robots.txt file.
c � � , U m - U m , U m PU m P
U m P
U m U O
U2 ] U m P! �N)�entries�groups�sitemaps�
default_entry�disallow_all� allow_all�set_url�last_checked��self�urls &&�'C:\Python315\\Lib\urllib\robotparser.py�__init__�RobotFileParser.__init__ sB � ���������
�!���!���������S����� c � � U O ! )z�Returns the time the robots.txt file was last fetched.
This is useful for long-running web spiders that need to
check for new robots.txt files periodically.
)r
�r s &r �mtime�RobotFileParser.mtime( s � � � � � r c �: � ] PHoUO 2 U m P! )zISets the time the robots.txt file was last fetched to the
current time.
)�timer
)r r s & r �modified�RobotFileParser.modified1 s � �
� �I�I�K��r c �z � Vm [ O O U2 Q* v U m U m P! )z,Sets the URL referring to a robots.txt file.:� � N)r �urllib�parse�urlsplit�host�pathr s &&r r �RobotFileParser.set_url9 s) � ���%�|�|�4�4�S�9�#�>��� �4�9r c � � [ O O U O 2 oUO 2 oU O UO
QQ2 O 2 2 P! [ O O c| oSO Q7 c P S m LGQSO t96: c Q6 c L LP S m L#QSO t96: c Q6 c L LP S m SO 2 Po=P! Po=hh 9 h)z4Reads the robots.txt URL and feeds it to the parser.zutf-8�surrogateescape� �� �X )� � )r �request�urlopenr �readr! �decode�
splitlines�error� HTTPError�coder
r �close)r �f�raw�errs & r r/ �RobotFileParser.read>