-
Notifications
You must be signed in to change notification settings - Fork 64
Unclear partial binding semantics #110
Comments
#96 also discusses this issue. |
Sorry, I missed that. So if I understand correctly from the conversation there, "as the matching proceeds, partial subpattern matches can do bindings that will persist even if the outer pattern fails". Should something like that be added to the PEP? |
It is undefined. All the implementation guarantees is that:
The same is true of sub-patterns in e.g. mapping, sequence, or class patterns. As I noted in the linked issue:
So, for example, this is guaranteed to work: match items:
case (x, .x):
print("Got a matching pair!") I agree that this could probably be clarified in the PEP somewhere. |
I see. I'm a bit surprised about it: I don't remember another part of Python where it's not clearly defined if some evaluation happens or not, and other related constructs like unpacking assignment where it's defined that you don't get any values assigned unless all are. But for the sake of separating things, I'll keep this issue related to "the decided partial binding behaviour should be explicit in the PEP" |
I agree with @dmoisset; I don't care much what the semantics are, but it should be clearly defined. Python isn't C and we shouldn't have undefined behavior. I suppose in most languages with pattern matching this problem doesn't really come up because the newly defined variables are in a new, separate scope, so they're invisible after the case block ends either way. |
Hm. Realistically what this would look like is guaranteeing that no bindings happen until the whole pattern succeeds (but before the guard is evaluated). Though well-defined, I'm not sure that it is really an improvement... but my opinions on this are not very strongly held. Honestly, I think that any code that relies on (possibly) partially-matched values isn't very well-written. But maybe there are examples that could persuade me otherwise. |
A definition like that (no bindings happen until the whole pattern succeeds, but before the guard) would be ideal for me, but any other will probably be ok, because as you say code relying on that is probably bad code. The problem with the set of variables assigned being undefined, from a user standpoint, is that I can not rely on those variables being set, but also I have to be careful about how I use those names, because they might be clobbered anyway. And that could change from python3.9.1 to python 3.9.2. The second problem is that cpython will have some behaviour (doesn't matter which), and someone will rely on it, and then other interpreters like pypy will have to implement the same because they want to be compatible, no matter what fun optimisations they can come up with :) Sidenote: I just realised that I lied about unpacking... even if |
Well, the set of variables that may be clobbered is unambiguous. The reason we don’t want to specify the exact rules is that they are complex, and small changes to a pattern may drastically change the behavior. (E.g. adding a * to a sequence pattern.) So users are better off not counting on variables being clobbered or not. |
That's a reasonable thing to add to the PEP. It doesn't commit us to trying subpatterns in a particular order (though other sections may) and it doesn't state the bindings must be made. (Maybe change "can" to "may"?) |
Just to give two examples to illustrate that the rules are not quite as easy as one might expect.
I feel this is also slightly related to the scoping issue #17, with the question whether captured variables would be limited in scope to the case handler. |
@Tobias-Kohn , the two examples you mention are explicitly covered in the PEP already: guards specify that the binding on the pattern is made before the condition is evaluated. The walrus says that the name is bound when the subpattern attached to it succeeds. |
ok, I see if I can submit a rephrase later today |
I've added a sample PR (note that this is against my personal fork of python/peps) for discussion and comments. If people here like this phrasing, I can submit this upstream: dmoisset/peps#3 |
Just as a note, I haven't moved the PR to python/peps yet because I'm sorting out some legal approval I require from my employer to contribute to open source and sign the CLA. (it shouldn't be a problem, other Python core devs already have gone through this same process and are helping me with this, but it may take a couple of days until they clear that) |
Sorted the legal approval, and pushed to PEPS repo |
Landed as python/peps@60a15e8 |
See python/cpython#22917 (comment) I see no justification for sloppy half matching, leaving unmatched variables assigned. |
Moving the discussion here. The way I see it, a case that isn't a match may still need to assign variables in certain situations, e.g.: case x if x > 0:
... In this situation I definitely want the semantics to be that a plain local variable If we were to erase the variable again when the guard is false, we should make sure that's also done when the guard raises an exception (like the Anyway, the simplest thing would be to leave I would also like to be able to step through pattern matching code (or at least through the guard) in a debugger and observe the variables. Finally it's not like we need to avoid stomping on a pre-existing That said I would not object if you contributed a patch implementing the semantics you prefer (in the context of Brandt's implementation), and in that case I'd be happy to amend the PEP to require this behavior. But my preference would be to keep the PEP as is and implement something better in the future. I don't want to have to spend an inordinate amount of effort creating an impeccable implementation, I just want pattern matching to land in 3.10. That kind of iteration is what we do for most feature development. (So, to be clear, I don't think this wins a beauty prize, but I also don't find the behavior allowed by the current proposal objectionable enough to disallow it.) |
You don't need to erase variables.
|
Incidentally, your code is buggy, of course, as it still leaves the value of More to the point, it is certainly not done by compiling the simplest possible example of a guard. Yes, it could all be done by leaving values on the stack, but in order to be usable in general we would also have to introduce some new bytecode instructions to fetch a specific values from the stack or store a value in the stack (and not necessarily on the top). So, in short in order for this to work in practice, we need enough power to establish basically an activation frame on the stack. All of this is not too difficult, of course, but it is not just easily solved with the existing infrastructure present in the interpreter. At this point, I would claim that it might even make more sense to introduce "shadow" local variables to hold intermediate values. Which, again, requires to modify the interpreter itself. |
Well, presumably the next case will use it. :)
Just brainstorming here. I think we could maybe make this work by stealing a trick from comprehensions and generator expressions. Really, we only need something like two new instructions:
Then, Mark's disassembly becomes:
This is nice in that it gives us many of the benefits of a register VM without any real architectural modification. I've already been toying around with similar ideas for caching other information (like the lengths of sequences and mappings), and the results are very promising. The only big problem that I see is that it limits the introspection capabilities of guards. Open question: whether these names, if bound, are present in >>> [locals() for i in [42]]
[{'.0': <tuple_iterator object at 0x7f7fb433c430>, 'i': 42}] |
And just to clarify: we wouldn't need the two new instructions. It's just that managing the temporary namespace using |
Yes, I thought along the same lines. However, I based my answer on the notion that these shadow variables would be completely hidden and inaccessible (like values on the stack), which then led to the idea that we would have to modify the code objects to account for these hidden variables. But using names like |
How does this tie with exception handling? I'm worried that using |
Good point. So we would also need something along the lines of a finally block. Bottom line: although it is possible to implement that semantic, it is not that simple and straight forward. Hence, we might want to concentrate on more important issues for the time being and leave this to a later version as Guido has indicated. |
I may have missed it, but it's not clear to me at which point are locals bound to the matched value in case of a partial match failure. For example:
What I could find is:
x
succeeded, but[x, 0]
failed. is there a bind?case
clause has succeeded and before the guard is evaluated. But it's not so clear if binding can happen while the pattern is evaluating.These things are a bit spread around. Perhaps a short "name binding semantics" could aggregate all these kinds of things together. I can help with that once I understand what's actually going on :)
The text was updated successfully, but these errors were encountered: