Front page | perl.perl5.porters |
Postings from March 2023
Re: strictness and "use 5.whatever"
Thread Previous
|
Thread Next
From:
demerphq
Date:
March 19, 2023 19:19
Subject:
Re: strictness and "use 5.whatever"
Message ID:
CANgJU+UpXKiMmZ6vOWgqHTh8Yf8Y6c8dV4B9OtZPw1iZ1nG10A@mail.gmail.com
Note that https://github.com/Perl/perl5/pull/20923 currently conflicts
with the deprecation categories patch that was merged. Also I plan to
release 5.37.10 tomorrow. Have you and Paul come to a mutually agreed
conclusion about this PR?
cheers,
Yves
On Sat, 11 Mar 2023 at 08:32, Zefram via perl5-porters <
perl5-porters@perl.org> wrote:
> Paul "LeoNerd" Evans wrote:
> > I can see (and overall like) the central change around op.c's lines
> > 7902-7906; that seems to be prettymuch what I had in mind to do, yes.
> > I'd be happy to extract that little part out and apply it.
>
> I'd better explain the relationship between the four patches. The patches
> perform four thematically related but logically independent changes,
> of which any subset could be meaningfully accepted. However there are
> textual dependencies between the patches. If you accept a subset that
> amounts to an initial subsequence of the train of four patches then
> no dependency issue arises; to accept a subset other than that would
> require a bit of editing, particularly of the documentation.
>
> There are other things that I'd be interested in doing in the same
> vicinity, such as replacing the feature-bundle bits in $^H with pure
> use of the individual feature bits (which semantically live in %^H
> but are rapidly accessible via cop_features). But that's an invisible
> implementation change, which can wait. The changes in this patch set are
> specifically the user-visible changes that it would be advantageous to get
> into 5.38. (With contentious changes freeze already past, I appreciate
> that you contending most of the changes pretty much rules that out.)
>
> On the subject of freeze dates: the first patch, which changes the meaning
> of "use v5.37" and the prospective meaning of "use v5.38", and the third
> patch, reversing the deprecation, are "user-visible changes to correctly
> functioning programs", so are subject to that freeze that's coming up
> in a few days. The other two patches, one purely documentation and the
> other adding a new facility to strict.pm, do not fall under that rubric.
>
> > consider making the cut-off version 5.35, rather than 5.37, so that
> > perl release 5.36.0 continues to have its current behaviour.
>
> The cutoff being 5.37 is precisely what achieves "use v5.36" retaining
> its current meaning. Try it out on an actual 5.36.0, actual 5.37.9, and
> blead with my proposed patches: on all three, "use v5.36" performs soft
> stricture enablement. Setting the cutoff to 5.35 would mean changing
> the effect of "use v5.36" from the soft stricture enablement that it
> currently has to the same hard stricture enablement that this patch
> gives to "use v5.37".
>
> > particular I don't see why any of the changes in strict.pm are needed
> > here.
>
> That's the implementation of what I said upthread about the explainability
> of the current behaviour of "use v5.16" et al. The motivation for that
> change is to provide strict.pm invocations that are precisely equivalent
> to the stricture aspect of version declarations lower than v5.37.
> The change goes a bit further, to provide full explicit control over
> the lexical state that already exists, including the ability to cancel
> the effects of all prior stricture declarations (which can be useful
> around eval).
>
> > Additionally, I see that PL_prevailing_version has been removed,
> > because it was only being used to support the version downgrade
> > ratchet warning.
>
> Not only because that was its only use, but also because it's
> intrinsically a bad idea. I quite agreed with your comment at
> <https://github.com/Perl/perl5/pull/19396> that it's a slippery slope
> that provides the temptation to make semantics depend directly on it.
>
> The `prevailing version' is additional lexical state, and therefore
> an additional thing that programmers need to know exists and need to
> manipulate. It isn't documented it in its full form; only the effect of
> changing it from >=v5.11 to <v5.11 is documented, which could have been
> achieved with just a one-bit flag. So the full PL_prevailing_version
> is sitting there, available to XS and providing obvious temptation,
> but unknown to programmers who don't read the source. Even if properly
> documented, though, it's a problem. Being available, it will inevitably
> get used, and all sorts of version thresholds will proliferate.
>
> You express concern that a version declaration should be equivalent
> to some series of more explicit pragmata. Well, PL_prevailing_version
> directly prevents there being such an equivalence. "use v5.34" doesn't
> just do the version check and the feature thing and the stricture thing,
> it also sets PL_prevailing_version to v5.34. If one wants to replicate
> its lexical effects, the feature and stricture pragmata aren't enough
> (even with a pragma that has the matching effect on strictures), one
> must also set PL_prevailing_version itself, and the only supported way
> to do that is "use v5.34".
>
> > We'd still verymuch like to keep that warning in
> > place
>
> Any form of the downgrade ratchet is a bad idea. There's nothing
> intrinsically wrong with having an island of old-style code in a
> generally new-style environment. It's no more wrong than having
> an island of new-style code in a generally old-style environment.
> (It would be more coherent, though still a bad idea, to prohibit *any*
> shadowing of version declaration by version declaration.)
>
> >where "reset a bunch of stuff" means turning back off all the
> >strict/warnings/feature bundles,
>
> The resetting from a version declaration has never had any effect of
> turning off warnings. The turning-off-stricture effect (for versions
> <v5.11) has also never been a full reset that would override explicit
> stricture enablement. It has only gone as far as soft disablement,
> cancelling the soft enablement from a higher version declaration.
>
> > These resets all work because
>
> What does it mean for these resets to "work"? I find this to be a
> slippery concept. The broad goal seems to be to enable a chunk of code
> written for one version bundle to be embedded in code written for a
> different version bundle, the intent being that the version declaration
> should be sufficient for the inner code to be interpreted correctly.
> But the details are tricky. Because the state that can be manipulated
> by version declarations can mostly also be manipulated by other pragmata,
> there are many interactions to consider. And because we want some kinds
> of lexical state to persist through a version declaration, the reset
> can't be entirely to a fixed lexical state.
>
> > because strict flags, warning
> >flags, and feature flags are all implemented as dedicated bits in some
> >special unique lexically-scoped set of storage in the interpreter,
>
> A relevant criterion, which I think you're hunting for here, is whether
> version declarations always set these bits of lexical state. If every
> version declaration sets the same state then you get a simple mental
> model that avoids surprises. What you seem to be getting at with
> "special unique lexically-scoped set of storage" is the idea that it's
> a well-defined portion of state that everyone understands belongs to
> version declarations.
>
> A big complication is that, for this mental model to work for the
> programmer, "always" has to apply not just to every version declaration
> as implemented by the current version, but also to version declarations
> as historically implemented. This doesn't require that all the state
> in question exist as a settable thing in all historical Perl versions.
> What it rquires is that, as soon as a new bit of state became settable
> at all, every version declaration had the effect of setting it.
> (Also, for basic stability, all version declarations for versions
> that predate it being settable must set it to the state matching the
> historical behaviour.) So an additional criterion for your "special
> unique lexically-scoped set of storage" is that it's new enough that
> it's always been under the control of version declarations.
>
> (The two complicating factors that I pointed out above, regarding how
> to interpret "these resets all work", correspond to two ways that the
> criterion I've just developed could in theory have been easily satisfied.
> If the state set by version declarations couldn't be manipulated in any
> other way then that would obviously make it a well-defined bit of state
> that's easy to have version declarations always set. And if we didn't
> want any lexical state to be preserved across a version declaration then
> the state being reset would be well-defined by being everything.)
>
> The aspects of lexical state that you list don't really achieve this
> ideal that "the reset works", precisely because they are not always
> set by version declarations. They're all more complicated than that.
> Warning state has only ever been set by version declarations >=5.35.
> Stricture is more complicated: on current Perls version declarations can
> be viewed as always setting a hidden `soft' stricture state, but taking a
> view that encompasses historical Perl versions it's not that consistent.
> Prior to Perl 5.15.6 some version declarations set the hard stricture
> state and some didn't touch stricture at all. Feature flags are the
> closest to achiving the ideal, the newer ones being always effectively
> set, but version declarations <5.9.5 didn't always clear the "say"
> feature, and version declarations <5.11 didn't always clear the
> "unicode_strings" feature. (Both of those also changed in Perl 5.15.6.)
>
> The proposed change of meaning for the stricture aspects of historical
> version declarations can be examined in terms of this criterion.
> Apart from its goal of freeing up $^H bits, the aim of the proposal seems
> to be to arrange for all version declarations to set the same range of
> lexical state, given that declarations of sufficiently new versions are
> going to set the hard stricture state. If that's a given then, to keep
> all version declarations setting the same state, it requires that they
> all set the hard stricture state, which is precisely what is proposed.
> Thus it's a proposal to help version declarations cleanly shadow each
> other, including downgrades, and thus undermines the deprecation of
> downgrading (across the stricture behaviour threshold) that was done in
> its name.
>
> But this analysis of the proposal only holds when looking only at the
> behaviour of version declarations on new Perls. When looking across
> Perl versions, the proposal obviously fails to keep the affected set of
> stricture state consistent, by making it inconsistent whether version
> declarations affect the hard stricture state. (For version declarations
> >=5.16 this would be a totally new inconsistency; for lower version
> declarations there was already some inconsistency when considering
> running on Perls <5.15.6.)
>
> There's some tension here between getting consistency between different
> version declarations running on one Perl and consistency of a single
> version declaration running on different Perl versions. My view is
> that the former hasn't historically been the case and is essentially
> unachievable, whereas the latter is an important backcompat issue.
> You seem to be pursuing the former at the expense of the latter.
>
> >As part of our umbrella of "making use VERSION simpler to explain", we
> >thought that if we forbid downgrading from a post-5.11 to a pre-5.11
> >version (the time at which strict happens), then it becomes a bit
> >easier to explain without needing to explain this reset.
>
> I reckon 5.11 is the wrong threshold to achieve easy explanation. Your
> "explaining the reset" thing amounts to establishing the range of lexical
> state that is set by a version declaration: it's easy to explain if the
> range of state is the same for all involved versions. More generally
> it's easy if the inner version declaration sets a superset of the state
> set by the outer version, and you're presuming that the range of state
> being set will monotonically increase with increasing declared version.
> But 5.11 doesn't bring in any change to the range of state being set.
> Under the current semantics, both "use v5.10" and "use v5.12" set the
> soft stricture state, so they fully shadow each other, so that's easy
> to explain.
>
> If declaring a new version (say 5.40) sets the hard stricture state and
> declaring historical versions still only sets the soft stricture state,
> then that does create a threshold at which there is a change in the
> range of state being set, and that threshold is somewhere around 5.39.
> It's downgrading across *that* threshold that one might want to forbid.
> It wouldn't require a deprecation, because you'd only be forbidding
> something that was impossible before anyway. The proposal to change
> historical version declarations to hard stricture setting is a way to
> avoid creating this threshold, and thus to avoid the perceived need to
> forbid downgrades (at any threshold) due to stricture behaviour.
>
> However, any proposed change with respect to historical version
> declarations fails at "making use VERSION simpler to explain". You can't
> make it simpler by making the behaviour more complicated, especially
> if the new explanation needs to include a full explanation of the old
> behaviour. We don't get to fully replace the behaviour of historical
> version declarations; we only get to add another Perl version's treatment
> of them. The minimum overall complexity is almost always achieved by
> adding behaviour that's the same as in the preceding version.
>
> > At that point it makes sense for builtin.pm, like
> >feature.pm, to have version bundles. At which point, it would feel
> >quite natural for `use VERSION` to activate those things too.
>
> That's certainly not a crazy thing to do.
>
> > They're just more functions in the lexical pad.
>
> Yes. Now you'd have a version declaration setting some bits of lexical
> namespace. That's preexisting lexical state that version declarations
> have never touched before, and you're not setting all of the lexical
> namespace or some delimited portion of it but a handful of arbitrary
> names within it. This certainly does pose some difficulty for deciding
> the semantics of shadowing version declarations.
>
> As a motivation for a downgrade ratchet, it has some merit with respect to
> the future versions to which these bundles are attached. The downgrades
> that are a problem are those that decrease the range of lexical state
> being set by a version declaration. This logic would lead one to forbid
> downgrades that decrease the set of implicit imports from builtin.
> This does not rationally motivate forbidding downgrades across 5.11, let
> alone all downgrades. The downgrades that are already possible are not a
> problem, and have no reason to be deprecated, with one exception: the 5.35
> threshold, which changes whether a version declaration affects warnings.
>
> However, prohibiting downgrades seems an overreaction.
>
> The bits of lexical state that are at issue, the referents of specific
> lexical subroutine names, is a type of state that is already familiar to
> code written for the older Perl versions. This same feature that makes it
> able to avoid the reset and persist past a later version declaration also
> makes it tractable to the inner code. I noted above that we generally
> want some kinds of lexical state to persist through a version declaration.
> The bindings of names are an excellent example of state that we do want
> to persist (except as specifically overridden by imports). Whereas v5.14
> code could well be surprised if it were parsed with "fc" being a keyword,
> and v5.26 code may be surprised if it's parsed with "^" being a strictly
> numeric operator, we don't have anything like those problems from "&true"
> being bound to a particular sub.
>
> I think the semantics should be that a version declaration leaves in
> place bindings that it's not specifically overriding as part of its
> bundle, even bindings that result from a prior version declaration.
> This seems totally workable and consistent with the principle of least
> astonishment. Another workable possibility, less good in my opinion,
> would be to try to determine which bindings came from prior version
> declarations, and shadow them with some kind of null pad entry that's
> equivalent to them having never been declared. We should pick one of
> these semantics, implement it, document it, and let it be.
>
> We could, however, do with supplying some kind of pragma or other
> declaration to explicitly evict bindings from the pad. My Lexical::Var
> module supplies such a facility (on the Perl versions where it works --
> I'm working on an update). This would have some value in combination
> with downgrading version declarations, but more generally have value
> where there's a tricky relationship between nested lexical scopes.
>
> The same logic that says that new lexical bindings aren't a problem
> for version declaration shadowing also suggest that the existing 5.35
> threshold, for whether warnings are set by a version declaration,
> isn't a problem. Lexical warning state has long existed, and was
> always previously preserved through version declarations. So it's not
> a problem for "use v5.34" that warnings might already be turned on, and
> specifically that they might have been turned on by a prior "use v5.36".
> Similarly, in Perl 5.14 it wasn't a problem that strictures persisted
> through "use v5.10" and that those strictures might have been enabled
> by a prior "use v5.12". The feature enablements that used to persist
> through low version declarations, though, were a bit of an issue.
>
> > To put it another way, we want
> >to prevent islands of lower versions from being allowed inside a sea of
> >a later version.
>
> You're stating this as if it's the motivation, when it's actually the
> conclusion that you've reached. The problem isn't islands of lower
> versions per se. The problem is the semantics of version-implied imports
> around version declaration shadowing, and you've concluded that we should
> avoid having to decide on semantics by prohibiting the shadowing that
> raises that issue.
>
> >Of course, there's nothing wrong with having them the other way around.
> >It's perfectly fine to bump upwards to a later version inside an island
> >within a sea of a lesser version.
>
> You're assuming here, as I brought out earlier, that higher version
> declarations will always set all of the lexical state that lower version
> declarations set. In the context of implicit imports, you're assuming
> that higher version declarations will always import everything that was
> imported by lower version declarations. But things like the "indirect"
> feature being removed from recent feature bundles show that this won't
> necessarily be the case.
>
> What happens when we decide that some implicitly-imported builtin
> sub is no longer good style, and remove it from new import bundles?
> Upgrading version declarations across such a threshold raises the same
> issues as downgrading across a threshold where imports were added.
> Do you forbid the upgrade? Do you have all future bundles import some
> unusable dummy sub under the name of the sub that's no longer approved?
>
> >The other reason I originally added it is because we don't want to keep
> >growing more feature bits for all of time.
>
> We have a pretty good generic treatment of feature bits. It doesn't
> seem like a problem to maintain a gradually-growing list of them.
>
> > It may be useful at
> >that point to stop tracking individual feature bits and instead just
> >say that those features are always enabled if the prevailing version is
> >greater than some boundary version when it first showed up.
>
> Eww. That would be gratuitous breakage of existing correct code.
>
> -zefram
>
--
perl -Mre=debug -e "/just|another|perl|hacker/"
Thread Previous
|
Thread Next