Changes between Version 5 and Version 6 of BluePrint/Messaging/Outbound
- Timestamp:
- 09/21/12 09:11:09 (12 years ago)
Legend:
- Unmodified
- Added
- Removed
- Modified
-
BluePrint/Messaging/Outbound
v5 v6 3 3 4 4 == Outbound Queue Performance == 5 === Issues === 5 6 We would like to ship out new messages in a timely manner, and also handle retrying failed messages, but avoid unnecessary overhead, and without doing time-consuming sends in request context where they will slow down response to the user. 6 7 … … 18 19 (Dominic points out that if each outbound send processes the entire queue including retries, then causing outbound sends could be used as a DoS attack. That works especially "well" if the sends are sure to fail and become retries.) 19 20 21 === Options === 22 Given that we want to queue up new messages rather than attempt a direct send before queuing (to avoid slow request response), and given that retries are a significant potential overhead, one option is to separate new message processing from retries. Some options: 23 * Use separate queues for new messages versus retries. After a run of the new message queue, each message will either be sent or will fail and be inserted in the retry queue, so -- except for messages inserted during the run -- the new message queue will be empty. This does not require a change to the queuing mechanism. Sub-options: 24 * Launch the new message queue asynchronously when each new message is posted. Here, there would be a periodic task just for the retry queue. 25 * Defer new message sends to a periodic job that runs sufficiently frequently to not cause user annoyance. Here, there would be periodic tasks for both new and retry messages. 26 * Mark retries and add a parameter that specifies whether retries should be processed on a given run. This requires a change to the queuing mechanism, and although retries would not be attempted during a new message run, they would still have to be iterated over. 27 28 Make sure queue runs do not overlap. 29 Must make sure that inserts while the queue is being processed are (thread) safe. 30 20 31 == Email Error Handling == 21 32 Web2py's Mail.send raises exceptions, contradicting its docstring. In particular, a common configuration error, failing to edit the example email sender, will cause a ticket when a user attempts to register. (Registration includes an email send even if no verification or approval is needed, as the user will get a welcome message in that case.) Since this is not a user error, but rather a misconfiguration that the site admin needs to deal with, it should not block registration nor show an error to the user. (See also ticket #439 -- we currently delete the user's temporary registration if there is an email error, but don't report anything to the site admin.) 22 33 23 We would prefer to have Web2py's Mail.send behavior fixed in Web2py, but this could be a backward-compatibility issue, and might take some time to resolve. So for now we'll subclass Mail (as MailS3) and show what we'd like done in Web2py. 34 We would prefer to have Web2py's Mail.send behavior fixed in Web2py, but this could be a backward-compatibility issue, and might take some time to resolve. So for now we'll subclass Mail (as MailS3) and show what we'd like done in Web2py. We can store our MailS3 instance in current.mail so it will be used by Web2py's Auth for those actions we don't override -- password change and username recovery. 24 35 25 36 Since we're subclassing Mail anyway, we can introduce other changes. In particular, we might want to use the outbound message queue rather than having the user wait for a direct send to be performed. This solves the issue #439 with configuration errors versus registration-related email. The user will not see any email errors, we can leave in their temporary registration, and the actual send during queue processing will log an error for the site admin to deal with. When the configuration is repaired, the message should be retried and sent without further intervention.
