email.policy
: Policy Objects¶
New in version 3.3.
Source code: Lib/email/policy.py
The email
package’s prime focus is the handling of email messages as
described by the various email and MIME RFCs. However, the general format of
email messages (a block of header fields each consisting of a name followed by
a colon followed by a value, the whole block followed by a blank line and an
arbitrary ‘body’), is a format that has found utility outside of the realm of
email. Some of these uses conform fairly closely to the main email RFCs, some
do not. Even when working with email, there are times when it is desirable to
break strict compliance with the RFCs, such as generating emails that
interoperate with email servers that do not themselves follow the standards, or
that implement extensions you want to use in ways that violate the
standards.
Policy objects give the email package the flexibility to handle all these disparate use cases.
A Policy
object encapsulates a set of attributes and methods that
control the behavior of various components of the email package during use.
Policy
instances can be passed to various classes and methods in the
email package to alter the default behavior. The settable values and their
defaults are described below.
There is a default policy used by all classes in the email package. For all of
the parser
classes and the related convenience functions, and for
the Message
class, this is the Compat32
policy, via its corresponding pre-defined instance compat32
. This
policy provides for complete backward compatibility (in some cases, including
bug compatibility) with the pre-Python3.3 version of the email package.
This default value for the policy keyword to
EmailMessage
is the EmailPolicy
policy, via
its pre-defined instance default
.
When a Message
or EmailMessage
object is created, it acquires a policy. If the message is created by a
parser
, a policy passed to the parser will be the policy used by
the message it creates. If the message is created by the program, then the
policy can be specified when it is created. When a message is passed to a
generator
, the generator uses the policy from the message by
default, but you can also pass a specific policy to the generator that will
override the one stored on the message object.
The default value for the policy keyword for the email.parser
classes
and the parser convenience functions will be changing in a future version of
Python. Therefore you should always specify explicitly which policy you want
to use when calling any of the classes and functions described in the
parser
module.
The first part of this documentation covers the features of Policy
, an
abstract base class that defines the features that are common to all
policy objects, including compat32
. This includes certain hook
methods that are called internally by the email package, which a custom policy
could override to obtain different behavior. The second part describes the
concrete classes EmailPolicy
and Compat32
, which implement
the hooks that provide the standard behavior and the backward compatible
behavior and features, respectively.
Policy
instances are immutable, but they can be cloned, accepting the
same keyword arguments as the class constructor and returning a new
Policy
instance that is a copy of the original but with the specified
attributes values changed.
As an example, the following code could be used to read an email message from a
file on disk and pass it to the system sendmail
program on a Unix system:
>>> from email import message_from_binary_file
>>> from email.generator import BytesGenerator
>>> from email import policy
>>> from subprocess import Popen, PIPE
>>> with open('mymsg.txt', 'rb') as f:
... msg = message_from_binary_file(f, policy=policy.default)
>>> p = Popen(['sendmail', msg['To'].addresses[0]], stdin=PIPE)
>>> g = BytesGenerator(p.stdin, policy=msg.policy.clone(linesep='\r\n'))
>>> g.flatten(msg)
>>> p.stdin.close()
>>> rc = p.wait()
Here we are telling BytesGenerator
to use the RFC
correct line separator characters when creating the binary string to feed into
sendmail's
stdin
, where the default policy would use \n
line
separators.
Some email package methods accept a policy keyword argument, allowing the
policy to be overridden for that method. For example, the following code uses
the as_bytes()
method of the msg object from
the previous example and writes the message to a file using the native line
separators for the platform on which it is running:
>>> import os
>>> with open('converted.txt', 'wb') as f:
... f.write(msg.as_bytes(policy=msg.policy.clone(linesep=os.linesep)))
17
Policy objects can also be combined using the addition operator, producing a policy object whose settings are a combination of the non-default values of the summed objects:
>>> compat_SMTP = policy.compat32.clone(linesep='\r\n')
>>> compat_strict = policy.compat32.clone(raise_on_defect=True)
>>> compat_strict_SMTP = compat_SMTP compat_strict
This operation is not commutative; that is, the order in which the objects are added matters. To illustrate:
>>> policy100 = policy.compat32.clone(max_line_length=100)
>>> policy80 = policy.compat32.clone(max_line_length=80)
>>> apolicy = policy100 policy80
>>> apolicy.max_line_length
80
>>> apolicy = policy80 policy100
>>> apolicy.max_line_length
100
-
class
email.policy.
Policy
(**kw)¶ This is the abstract base class for all policy classes. It provides default implementations for a couple of trivial methods, as well as the implementation of the immutability property, the
clone()
method, and the constructor semantics.The constructor of a policy class can be passed various keyword arguments. The arguments that may be specified are any non-method properties on this class, plus any additional non-method properties on the concrete class. A value specified in the constructor will override the default value for the corresponding attribute.
This class defines the following properties, and thus values for the following may be passed in the constructor of any policy class:
-
max_line_length
¶ The maximum length of any line in the serialized output, not counting the end of line character(s). Default is 78, per RFC 5322. A value of
0
orNone
indicates that no line wrapping should be done at all.
-
linesep
¶ The string to be used to terminate lines in serialized output. The default is
\n
because that’s the internal end-of-line discipline used by Python, though\r\n
is required by the RFCs.
-
cte_type
¶ Controls the type of Content Transfer Encodings that may be or are required to be used. The possible values are:
7bit
all data must be “7 bit clean” (ASCII-only). This means that where necessary data will be encoded using either quoted-printable or base64 encoding.
8bit
data is not constrained to be 7 bit clean. Data in headers is still required to be ASCII-only and so will be encoded (see
fold_binary()
andutf8
below for exceptions), but body parts may use the8bit
CTE.A
cte_type
value of8bit
only works withBytesGenerator
, notGenerator
, because strings cannot contain binary data. If aGenerator
is operating under a policy that specifiescte_type=8bit
, it will act as ifcte_type
is7bit
.
-
raise_on_defect
¶ If
True
, any defects encountered will be raised as errors. IfFalse
(the default), defects will be passed to theregister_defect()
method.
-
mangle_from_
¶ If
True
, lines starting with “From “ in the body are escaped by putting a>
in front of them. This parameter is used when the message is being serialized by a generator. Default:False
.New in version 3.5: The mangle_from_ parameter.
-
message_factory
¶ A factory function for constructing a new empty message object. Used by the parser when building messages. Defaults to
None
, in which caseMessage
is used.New in version 3.6.
The following
Policy
method is intended to be called by code using the email library to create policy instances with custom settings:-
clone
(**kw)¶ Return a new
Policy
instance whose attributes have the same values as the current instance, except where those attributes are given new values by the keyword arguments.
The remaining
Policy
methods are called by the email package code, and are not intended to be called by an application using the email package. A custom policy must implement all of these methods.-
handle_defect
(obj, defect)¶ Handle a defect found on obj. When the email package calls this method, defect will always be a subclass of
Defect
.The default implementation checks the
raise_on_defect
flag. If it isTrue
, defect is raised as an exception. If it isFalse
(the default), obj and defect are passed toregister_defect()
.
-
register_defect
(obj, defect)¶ Register a defect on obj. In the email package, defect will always be a subclass of
Defect
.The default implementation calls the
append
method of thedefects
attribute of obj. When the email package callshandle_defect
, obj will normally have adefects
attribute that has anappend
method. Custom object types used with the email package (for example, customMessage
objects) should also provide such an attribute, otherwise defects in parsed messages will raise unexpected errors.
-
header_max_count
(name)¶ Return the maximum allowed number of headers named name.
Called when a header is added to an
EmailMessage
orMessage
object. If the returned value is not0
orNone
, and there are already a number of headers with the name name greater than or equal to the value returned, aValueError
is raised.Because the default behavior of
Message.__setitem__
is to append the value to the list of headers, it is easy to create duplicate headers without realizing it. This method allows certain headers to be limited in the number of instances of that header that may be added to aMessage
programmatically. (The limit is not observed by the parser, which will faithfully produce as many headers as exist in the message being parsed.)The default implementation returns
None
for all header names.
-
header_source_parse
(sourcelines)¶ The email package calls this method with a list of strings, each string ending with the line separation characters found in the source being parsed. The first line includes the field header name and separator. All whitespace in the source is preserved. The method should return the
(name, value)
tuple that is to be stored in theMessage
to represent the parsed header.If an implementation wishes to retain compatibility with the existing email package policies, name should be the case preserved name (all characters up to the ‘
:
’ separator), while value should be the unfolded value (all line separator characters removed, but whitespace kept intact), stripped of leading whitespace.sourcelines may contain surrogateescaped binary data.
There is no default implementation
-
header_store_parse
(name, value)¶ The email package calls this method with the name and value provided by the application program when the application program is modifying a
Message
programmatically (as opposed to aMessage
created by a parser). The method should return the(name, value)
tuple that is to be stored in theMessage
to represent the header.If an implementation wishes to retain compatibility with the existing email package policies, the name and value should be strings or string subclasses that do not change the content of the passed in arguments.
There is no default implementation
-
header_fetch_parse
(name, value)¶ The email package calls this method with the name and value currently stored in the
Message
when that header is requested by the application program, and whatever the method returns is what is passed back to the application as the value of the header being retrieved. Note that there may be more than one header with the same name stored in theMessage
; the method is passed the specific name and value of the header destined to be returned to the application.value may contain surrogateescaped binary data. There should be no surrogateescaped binary data in the value returned by the method.
There is no default implementation
-
fold
(name, value)¶ The email package calls this method with the name and value currently stored in the
Message
for a given header. The method should return a string that represents that header “folded” correctly (according to the policy settings) by composing the name with the value and insertinglinesep
characters at the appropriate places. See RFC 5322 for a discussion of the rules for folding email headers.value may contain surrogateescaped binary data. There should be no surrogateescaped binary data in the string returned by the method.
-
-
class
email.policy.
EmailPolicy
(**kw)¶ This concrete
Policy
provides behavior that is intended to be fully compliant with the current email RFCs. These include (but are not limited to) RFC 5322, RFC 2047, and the current MIME RFCs.This policy adds new header parsing and folding algorithms. Instead of simple strings, headers are
str
subclasses with attributes that depend on the type of the field. The parsing and folding algorithm fully implement RFC 2047 and RFC 5322.The default value for the
message_factory
attribute isEmailMessage
.In addition to the settable attributes listed above that apply to all policies, this policy adds the following additional attributes:
New in version 3.6: 1
-
utf8
¶ If
False
, follow RFC 5322, supporting non-ASCII characters in headers by encoding them as “encoded words”. IfTrue
, follow RFC 6532 and useutf-8
encoding for headers. Messages formatted in this way may be passed to SMTP servers that support theSMTPUTF8
extension (RFC 6531).
-
refold_source
¶ If the value for a header in the
Message
object originated from aparser
(as opposed to being set by a program), this attribute indicates whether or not a generator should refold that value when transforming the message back into serialized form. The possible values are:none
all source values use original folding
long
source values that have any line that is longer than
max_line_length
will be refoldedall
all values are refolded.
The default is
long
.
-
header_factory
¶ A callable that takes two arguments,
name
andvalue
, wherename
is a header field name andvalue
is an unfolded header field value, and returns a string subclass that represents that header. A defaultheader_factory
(seeheaderregistry
) is provided that supports custom parsing for the various address and date RFC 5322 header field types, and the major MIME header field stypes. Support for additional custom parsing will be added in the future.
-
content_manager
¶ An object with at least two methods: get_content and set_content. When the
get_content()
orset_content()
method of anEmailMessage
object is called, it calls the corresponding method of this object, passing it the message object as its first argument, and any arguments or keywords that were passed to it as additional arguments. By defaultcontent_manager
is set toraw_data_manager
.New in version 3.4.
The class provides the following concrete implementations of the abstract methods of
Policy
:-
header_max_count
(name)¶ Returns the value of the
max_count
attribute of the specialized class used to represent the header with the given name.
-
header_source_parse
(sourcelines)¶ The name is parsed as everything up to the ‘
:
’ and returned unmodified. The value is determined by stripping leading whitespace off the remainder of the first line, joining all subsequent lines together, and stripping any trailing carriage return or linefeed characters.
-
header_store_parse
(name, value)¶ The name is returned unchanged. If the input value has a
name
attribute and it matches name ignoring case, the value is returned unchanged. Otherwise the name and value are passed toheader_factory
, and the resulting header object is returned as the value. In this case aValueError
is raised if the input value contains CR or LF characters.
-
header_fetch_parse
(name, value)¶ If the value has a
name
attribute, it is returned to unmodified. Otherwise the name, and the value with any CR or LF characters removed, are passed to theheader_factory
, and the resulting header object is returned. Any surrogateescaped bytes get turned into the unicode unknown-character glyph.
-
fold
(name, value)¶ Header folding is controlled by the
refold_source
policy setting. A value is considered to be a ‘source value’ if and only if it does not have aname
attribute (having aname
attribute means it is a header object of some sort). If a source value needs to be refolded according to the policy, it is converted into a header object by passing the name and the value with any CR and LF characters removed to theheader_factory
. Folding of a header object is done by calling itsfold
method with the current policy.Source values are split into lines using
splitlines()
. If the value is not to be refolded, the lines are rejoined using thelinesep
from the policy and returned. The exception is lines containing non-ascii binary data. In that case the value is refolded regardless of therefold_source
setting, which causes the binary data to be CTE encoded using theunknown-8bit
charset.
-
fold_binary
(name, value)¶ The same as
fold()
ifcte_type
is7bit
, except that the returned value is bytes.If
cte_type
is8bit
, non-ASCII binary data is converted back into bytes. Headers with binary data are not refolded, regardless of therefold_header
setting, since there is no way to know whether the binary data consists of single byte characters or multibyte characters.
-
The following instances of EmailPolicy
provide defaults suitable for
specific application domains. Note that in the future the behavior of these
instances (in particular the HTTP
instance) may be adjusted to conform even
more closely to the RFCs relevant to their domains.
-
email.policy.
default
¶ An instance of
EmailPolicy
with all defaults unchanged. This policy uses the standard Python\n
line endings rather than the RFC-correct\r\n
.
-
email.policy.
SMTP
¶ Suitable for serializing messages in conformance with the email RFCs. Like
default
, but withlinesep
set to\r\n
, which is RFC compliant.
-
email.policy.
SMTPUTF8
¶ The same as
SMTP
except thatutf8
isTrue
. Useful for serializing messages to a message store without using encoded words in the headers. Should only be used for SMTP transmission if the sender or recipient addresses have non-ASCII characters (thesmtplib.SMTP.send_message()
method handles this automatically).
-
email.policy.
HTTP
¶ Suitable for serializing headers with for use in HTTP traffic. Like
SMTP
except thatmax_line_length
is set toNone
(unlimited).
-
email.policy.
strict
¶ Convenience instance. The same as
default
except thatraise_on_defect
is set toTrue
. This allows any policy to be made strict by writing:somepolicy policy.strict
With all of these EmailPolicies
, the effective API of
the email package is changed from the Python 3.2 API in the following ways:
Setting a header on a
Message
results in that header being parsed and a header object created.Fetching a header value from a
Message
results in that header being parsed and a header object created and returned.Any header object, or any header that is refolded due to the policy settings, is folded using an algorithm that fully implements the RFC folding algorithms, including knowing where encoded words are required and allowed.
From the application view, this means that any header obtained through the
EmailMessage
is a header object with extra
attributes, whose string value is the fully decoded unicode value of the
header. Likewise, a header may be assigned a new value, or a new header
created, using a unicode string, and the policy will take care of converting
the unicode string into the correct RFC encoded form.
The header objects and their attributes are described in
headerregistry
.
-
class
email.policy.
Compat32
(**kw)¶ This concrete
Policy
is the backward compatibility policy. It replicates the behavior of the email package in Python 3.2. Thepolicy
module also defines an instance of this class,compat32
, that is used as the default policy. Thus the default behavior of the email package is to maintain compatibility with Python 3.2.The following attributes have values that are different from the
Policy
default:-
mangle_from_
¶ The default is
True
.
The class provides the following concrete implementations of the abstract methods of
Policy
:-
header_source_parse
(sourcelines)¶ The name is parsed as everything up to the ‘
:
’ and returned unmodified. The value is determined by stripping leading whitespace off the remainder of the first line, joining all subsequent lines together, and stripping any trailing carriage return or linefeed characters.
-
header_store_parse
(name, value)¶ The name and value are returned unmodified.
-
header_fetch_parse
(name, value)¶ If the value contains binary data, it is converted into a
Header
object using theunknown-8bit
charset. Otherwise it is returned unmodified.
-
fold
(name, value)¶ Headers are folded using the
Header
folding algorithm, which preserves existing line breaks in the value, and wraps each resulting line to themax_line_length
. Non-ASCII binary data are CTE encoded using theunknown-8bit
charset.
-
fold_binary
(name, value)¶ Headers are folded using the
Header
folding algorithm, which preserves existing line breaks in the value, and wraps each resulting line to themax_line_length
. Ifcte_type
is7bit
, non-ascii binary data is CTE encoded using theunknown-8bit
charset. Otherwise the original source header is used, with its existing line breaks and any (RFC invalid) binary data it may contain.
-
-
email.policy.
compat32
¶ An instance of
Compat32
, providing backward compatibility with the behavior of the email package in Python 3.2.
Footnotes
- 1
Originally added in 3.3 as a provisional feature.