Skip to content

Commit a33df31

Browse files
bitdancerbitdancer
committed
#21795: advertise 8BITMIME if decode_data is False.
Patch by Milan Oberkirch, with a few updates. This changeset also tweaks the smtpd and whatsnew docs for smtpd into what should be the final form for the 3.5 release.
1 parent 0d905d4 commit a33df31

5 files changed

Lines changed: 219 additions & 97 deletions

File tree

‎Doc/library/smtpd.rst‎

Lines changed: 35 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -40,20 +40,27 @@ SMTPServer Objects
4040
accepted in a ``DATA`` command. A value of ``None`` or ``0`` means no
4141
limit.
4242

43-
*enable_SMTPUTF8* determins whether the ``SMTPUTF8`` extension (as defined
44-
in :RFC:`6531`) should be enabled. The default is ``False``. If
45-
*enable_SMTPUTF* is set to ``True``, the :meth:`process_smtputf8_message`
46-
method must be defined. A :exc:`ValueError` is raised if both
47-
*enable_SMTPUTF8* and *decode_data* are set to ``True`` at the same time.
43+
*map* is the socket map to use for connections (an initially empty
44+
dictionary is a suitable value). If not specified the :mod:`asyncore`
45+
global socket map is used.
4846

49-
A dictionary can be specified in *map* to avoid using a global socket map.
47+
*enable_SMTPUTF8* determins whether the ``SMTPUTF8`` extension (as defined
48+
in :RFC:`6531`) should be enabled. The default is ``False``. If set to
49+
``True``, *decode_data* must be ``False`` (otherwise an error is raised).
50+
When ``True``, ``SMTPUTF8`` is accepted as a parameter to the ``MAIL``
51+
command and when present is passed to :meth:`process_message` in the
52+
``kwargs['mail_options']`` list.
5053

5154
*decode_data* specifies whether the data portion of the SMTP transaction
5255
should be decoded using UTF-8. The default is ``True`` for backward
53-
compatibility reasons, but will change to ``False`` in Python 3.6. Specify
54-
the keyword value explicitly to avoid the :exc:`DeprecationWarning`.
56+
compatibility reasons, but will change to ``False`` in Python 3.6; specify
57+
the keyword value explicitly to avoid the :exc:`DeprecationWarning`. When
58+
*decode_data* is set to ``False`` the server advertises the ``8BITMIME``
59+
extension (:rfc:`6152`), accepts the ``BODY=8BITMIME`` parameter to
60+
the ``MAIL`` command, and when present passes it to :meth:`process_message`
61+
in the ``kwargs['mail_options']`` list.
5562

56-
.. method:: process_message(peer, mailfrom, rcpttos, data)
63+
.. method:: process_message(peer, mailfrom, rcpttos, data, **kwargs)
5764

5865
Raise a :exc:`NotImplementedError` exception. Override this in subclasses to
5966
do something useful with this message. Whatever was passed in the
@@ -67,34 +74,39 @@ SMTPServer Objects
6774
argument will be a unicode string. If it is set to ``False``, it
6875
will be a bytes object.
6976

70-
Return ``None`` to request a normal ``250 Ok`` response; otherwise
71-
return the desired response string in :RFC:`5321` format.
77+
*kwargs* is a dictionary containing additional information. It is empty
78+
unless at least one of ``decode_data=False`` or ``enable_SMTPUTF8=True``
79+
was given as an init parameter, in which case it contains the following
80+
keys:
81+
82+
*mail_options*:
83+
a list of all received parameters to the ``MAIL``
84+
command (the elements are uppercase strings; example:
85+
``['BODY=8BITMIME', 'SMTPUTF8']``).
7286

73-
.. method:: process_smtputf8_message(peer, mailfrom, rcpttos, data)
87+
*rcpt_options*:
88+
same as *mail_options* but for the ``RCPT`` command.
89+
Currently no ``RCPT TO`` options are supported, so for now
90+
this will always be an empty list.
7491

75-
Raise a :exc:`NotImplementedError` exception. Override this in
76-
subclasses to do something useful with messages when *enable_SMTPUTF8*
77-
has been set to ``True`` and the SMTP client requested ``SMTPUTF8``,
78-
since this method is called rather than :meth:`process_message` when the
79-
client actively requests ``SMTPUTF8``. The *data* argument will always
80-
be a bytes object, and any non-``None`` return value should conform to
81-
:rfc:`6531`; otherwise, the API is the same as for
82-
:meth:`process_message`.
92+
Return ``None`` to request a normal ``250 Ok`` response; otherwise
93+
return the desired response string in :RFC:`5321` format.
8394

8495
.. attribute:: channel_class
8596

8697
Override this in subclasses to use a custom :class:`SMTPChannel` for
8798
managing SMTP clients.
8899

89-
.. versionchanged:: 3.4
90-
The *map* argument was added.
100+
.. versionadded:: 3.4
101+
The *map* constructor argument.
91102

92103
.. versionchanged:: 3.5
93104
*localaddr* and *remoteaddr* may now contain IPv6 addresses.
94105

95106
.. versionadded:: 3.5
96107
the *decode_data* and *enable_SMTPUTF8* constructor arguments, and the
97-
:meth:`process_smtputf8_message` method.
108+
*kwargs* argument to :meth:`process_message` when one or more of these is
109+
specified.
98110

99111

100112
DebuggingServer Objects

‎Doc/whatsnew/3.5.rst‎

Lines changed: 17 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -468,16 +468,28 @@ smtpd
468468
transaction is decoded using the ``utf-8`` codec or is instead provided to
469469
:meth:`~smtpd.SMTPServer.process_message` as a byte string. The default
470470
is ``True`` for backward compatibility reasons, but will change to ``False``
471-
in Python 3.6. (Contributed by Maciej Szulik in :issue:`19662`.)
471+
in Python 3.6. If *decode_data* is set to ``False``, the
472+
:meth:`~smtpd.SMTPServer.process_message` method must be prepared to accept
473+
keyword arguments. (Contributed by Maciej Szulik in :issue:`19662`.)
474+
475+
* :class:`~smtpd.SMTPServer` now advertises the ``8BITMIME`` extension
476+
(:rfc:`6152`) if if *decode_data* has been set ``True``. If the client
477+
specifies ``BODY=8BITMIME`` on the ``MAIL`` command, it is passed to
478+
:meth:`~smtpd.SMTPServer.process_message` via the ``mail_options`` keyword.
479+
(Contributed by Milan Oberkirch and R. David Murray in :issue:`21795`.)
480+
481+
* :class:`~smtpd.SMTPServer` now supports the ``SMTPUTF8`` extension
482+
(:rfc:`6531`: Internationalized Email). If the client specified ``SMTPUTF8
483+
BODY=8BITMIME`` on the ``MAIL`` command, they are passed to
484+
:meth:`~smtpd.SMTPServer.process_message` via the ``mail_options`` keyword.
485+
It is the responsibility of the :meth:`~smtpd.SMTPServer.process_message`
486+
method to correctly handle the ``SMTPUTF8`` data. (Contributed by Milan
487+
Oberkirch in :issue:`21725`.)
472488

473489
* It is now possible to provide, directly or via name resolution, IPv6
474490
addresses in the :class:`~smtpd.SMTPServer` constructor, and have it
475491
successfully connect. (Contributed by Milan Oberkirch in :issue:`14758`.)
476492

477-
* :mod:`~smtpd.SMTPServer` now supports :rfc:`6531` via the *enable_SMTPUTF8*
478-
constructor argument and a user-provided
479-
:meth:`~smtpd.SMTPServer.process_smtputf8_message` method.
480-
481493
smtplib
482494
-------
483495

‎Lib/smtpd.py‎

Lines changed: 56 additions & 48 deletions
Original file line numberDiff line numberDiff line change
@@ -381,10 +381,13 @@ def found_terminator(self):
381381
data.append(text)
382382
self.received_data = self._newline.join(data)
383383
args = (self.peer, self.mailfrom, self.rcpttos, self.received_data)
384-
if self.require_SMTPUTF8:
385-
status = self.smtp_server.process_smtputf8_message(*args)
386-
else:
387-
status = self.smtp_server.process_message(*args)
384+
kwargs = {}
385+
if not self._decode_data:
386+
kwargs = {
387+
'mail_options': self.mail_options,
388+
'rcpt_options': self.rcpt_options,
389+
}
390+
status = self.smtp_server.process_message(*args, **kwargs)
388391
self._set_post_data_state()
389392
if not status:
390393
self.push('250 OK')
@@ -419,8 +422,9 @@ def smtp_EHLO(self, arg):
419422
if self.data_size_limit:
420423
self.push('250-SIZE %s' % self.data_size_limit)
421424
self.command_size_limits['MAIL'] += 26
422-
if self.enable_SMTPUTF8:
425+
if not self._decode_data:
423426
self.push('250-8BITMIME')
427+
if self.enable_SMTPUTF8:
424428
self.push('250-SMTPUTF8')
425429
self.command_size_limits['MAIL'] += 10
426430
self.push('250 HELP')
@@ -454,11 +458,15 @@ def _getaddr(self, arg):
454458
return address.addr_spec, rest
455459

456460
def _getparams(self, params):
457-
# Return any parameters that appear to be syntactically valid according
458-
# to RFC 1869, ignore all others. (Postel rule: accept what we can.)
459-
params = [param.split('=', 1) if '=' in param else (param, True)
460-
for param in params.split()]
461-
return {k: v for k, v in params if k.isalnum()}
461+
# Return params as dictionary. Return None if not all parameters
462+
# appear to be syntactically valid according to RFC 1869.
463+
result = {}
464+
for param in params:
465+
param, eq, value = param.partition('=')
466+
if not param.isalnum() or eq and not value:
467+
return None
468+
result[param] = value if eq else True
469+
return result
462470

463471
def smtp_HELP(self, arg):
464472
if arg:
@@ -508,7 +516,7 @@ def smtp_VRFY(self, arg):
508516

509517
def smtp_MAIL(self, arg):
510518
if not self.seen_greeting:
511-
self.push('503 Error: send HELO first');
519+
self.push('503 Error: send HELO first')
512520
return
513521
print('===> MAIL', arg, file=DEBUGSTREAM)
514522
syntaxerr = '501 Syntax: MAIL FROM: <address>'
@@ -528,18 +536,23 @@ def smtp_MAIL(self, arg):
528536
if self.mailfrom:
529537
self.push('503 Error: nested MAIL command')
530538
return
531-
params = self._getparams(params.upper())
539+
self.mail_options = params.upper().split()
540+
params = self._getparams(self.mail_options)
532541
if params is None:
533542
self.push(syntaxerr)
534543
return
535-
body = params.pop('BODY', '7BIT')
536-
if self.enable_SMTPUTF8 and params.pop('SMTPUTF8', False):
537-
if body != '8BITMIME':
538-
self.push('501 Syntax: MAIL FROM: <address>'
539-
' [BODY=8BITMIME SMTPUTF8]')
544+
if not self._decode_data:
545+
body = params.pop('BODY', '7BIT')
546+
if body not in ['7BIT', '8BITMIME']:
547+
self.push('501 Error: BODY can only be one of 7BIT, 8BITMIME')
540548
return
541-
else:
549+
if self.enable_SMTPUTF8:
550+
smtputf8 = params.pop('SMTPUTF8', False)
551+
if smtputf8 is True:
542552
self.require_SMTPUTF8 = True
553+
elif smtputf8 is not False:
554+
self.push('501 Error: SMTPUTF8 takes no arguments')
555+
return
543556
size = params.pop('SIZE', None)
544557
if size:
545558
if not size.isdigit():
@@ -574,16 +587,16 @@ def smtp_RCPT(self, arg):
574587
if not address:
575588
self.push(syntaxerr)
576589
return
577-
if params:
578-
if self.extended_smtp:
579-
params = self._getparams(params.upper())
580-
if params is None:
581-
self.push(syntaxerr)
582-
return
583-
else:
584-
self.push(syntaxerr)
585-
return
586-
if params and len(params.keys()) > 0:
590+
if not self.extended_smtp and params:
591+
self.push(syntaxerr)
592+
return
593+
self.rcpt_options = params.upper().split()
594+
params = self._getparams(self.rcpt_options)
595+
if params is None:
596+
self.push(syntaxerr)
597+
return
598+
# XXX currently there are no options we recognize.
599+
if len(params.keys()) > 0:
587600
self.push('555 RCPT TO parameters not recognized or not implemented')
588601
return
589602
self.rcpttos.append(address)
@@ -667,7 +680,7 @@ def handle_accepted(self, conn, addr):
667680
self._decode_data)
668681

669682
# API for "doing something useful with the message"
670-
def process_message(self, peer, mailfrom, rcpttos, data):
683+
def process_message(self, peer, mailfrom, rcpttos, data, **kwargs):
671684
"""Override this abstract method to handle messages from the client.
672685
673686
peer is a tuple containing (ipaddr, port) of the client that made the
@@ -685,21 +698,16 @@ def process_message(self, peer, mailfrom, rcpttos, data):
685698
containing a `.' followed by other text has had the leading dot
686699
removed.
687700
688-
This function should return None for a normal `250 Ok' response;
689-
otherwise, it should return the desired response string in RFC 821
690-
format.
691-
692-
"""
693-
raise NotImplementedError
694-
695-
# API for processing messeges needing Unicode support (RFC 6531, RFC 6532).
696-
def process_smtputf8_message(self, peer, mailfrom, rcpttos, data):
697-
"""Same as ``process_message`` but for messages for which the client
698-
has sent the SMTPUTF8 parameter with the MAIL command (see the
699-
enable_SMTPUTF8 parameter of the constructor).
701+
kwargs is a dictionary containing additional information. It is empty
702+
unless decode_data=False or enable_SMTPUTF8=True was given as init
703+
parameter, in which case ut will contain the following keys:
704+
'mail_options': list of parameters to the mail command. All
705+
elements are uppercase strings. Example:
706+
['BODY=8BITMIME', 'SMTPUTF8'].
707+
'rcpt_options': same, for the rcpt command.
700708
701709
This function should return None for a normal `250 Ok' response;
702-
otherwise, it should return the desired response string in RFC 6531
710+
otherwise, it should return the desired response string in RFC 821
703711
format.
704712
705713
"""
@@ -725,13 +733,13 @@ def _print_message_content(self, peer, data):
725733
line = repr(line)
726734
print(line)
727735

728-
def process_message(self, peer, mailfrom, rcpttos, data):
736+
def process_message(self, peer, mailfrom, rcpttos, data, **kwargs):
729737
print('---------- MESSAGE FOLLOWS ----------')
730-
self._print_message_content(peer, data)
731-
print('------------ END MESSAGE ------------')
732-
733-
def process_smtputf8_message(self, peer, mailfrom, rcpttos, data):
734-
print('----- SMTPUTF8 MESSAGE FOLLOWS ------')
738+
if kwargs:
739+
if kwargs.get('mail_options'):
740+
print('mail options: %s' % kwargs['mail_options'])
741+
if kwargs.get('rcpt_options'):
742+
print('rcpt options: %s\n' % kwargs['rcpt_options'])
735743
self._print_message_content(peer, data)
736744
print('------------ END MESSAGE ------------')
737745

0 commit comments

Comments
 (0)