develooper Front page | perl.perl5.changes | Postings from August 2022

[Perl/perl5] b59baa: loc_tools: Create function, refactor

From:
Karl Williamson via perl5-changes
Date:
August 11, 2022 17:59
Subject:
[Perl/perl5] b59baa: loc_tools: Create function, refactor
Message ID:
Perl/perl5/push/refs/heads/smoke-me/khw-locale4/2d0e20-334448@github.com
  Branch: refs/heads/smoke-me/khw-locale4
  Home:   https://github.com/Perl/perl5
  Commit: b59baa305c866e17b153044171a14d05a6b9bbb7
      https://github.com/Perl/perl5/commit/b59baa305c866e17b153044171a14d05a6b9bbb7
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M t/loc_tools.pl

  Log Message:
  -----------
  loc_tools: Create function, refactor

category_excluded() is a better name for what the current name
'is_category_valid" does.  And it turns out that we need to use this
function on (those few) systems that don't have LC_CTYPE to avoid
executing code that depends on LC_CTYPE.

And we do need an improved is_category_valid() for other checks.


  Commit: 94342c09bc8a13e55ca0ac2f82d181bfe101526a
      https://github.com/Perl/perl5/commit/94342c09bc8a13e55ca0ac2f82d181bfe101526a
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M t/loc_tools.pl

  Log Message:
  -----------
  loc_tools.pl: Move code in file

This is for future commits to call after it is defined


  Commit: 307a037f89d837d40ff33dd94917481a6d7021c6
      https://github.com/Perl/perl5/commit/307a037f89d837d40ff33dd94917481a6d7021c6
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M t/loc_tools.pl

  Log Message:
  -----------
  t/loc_tools.pl: Print better failing test numbers

This test module is 'required' from other perl test files.  If it
encounters an error, it doesn't know how to report it.  What it does is
call fail() if available, and a home-grown one otherwise.  Prior to this
commit the home-grown version just made all tests number 0.  This commit
changes that to make them sequentially numbered from a very high
starting one, so as not to interfere with the outer calls.


  Commit: 31bfa140e95a4917ac8c1562da7b47d97749d599
      https://github.com/Perl/perl5/commit/31bfa140e95a4917ac8c1562da7b47d97749d599
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M t/loc_tools.pl

  Log Message:
  -----------
  t/loc_tools.pl: Refactor _trylocale()

This function is used to see if a locale actually works on the current
platform.  I was not fully aware of the glitches if a category is in one
locale, and LC_CTYPE is in another.  This makes sure they are both the
same; and this results in some simplification.


  Commit: fc975970f38fff8b4578978a08515fe40d29533b
      https://github.com/Perl/perl5/commit/fc975970f38fff8b4578978a08515fe40d29533b
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M t/loc_tools.pl

  Log Message:
  -----------
  t/loc_tools.pl: Fail earlier

Move the code that returns failure into the loop, so won't keep
iterating if failure is going to happen anyway.


  Commit: 54a0e37a4041163b87ea86334e637d713e826c9d
      https://github.com/Perl/perl5/commit/54a0e37a4041163b87ea86334e637d713e826c9d
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M t/loc_tools.pl

  Log Message:
  -----------
  t/loc_tools.pl: Add checks

This verifies that we can restore a locale that we were previously in,
and makes sure that we don't stay in a locale that doesn't work well.
Doing so has led to crashes.


  Commit: 36489ca8235805675864dbaef1f4c470106b4577
      https://github.com/Perl/perl5/commit/36489ca8235805675864dbaef1f4c470106b4577
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M t/loc_tools.pl

  Log Message:
  -----------
  t/loc_tools.pl: Don't allow commas in locale names

This was a problem in some buggy Windows versions.  This addition keeps
locales with this bug from being tested.


  Commit: 4201e77ac2ed0f97eabef1b2c80b11cdec17d4c0
      https://github.com/Perl/perl5/commit/4201e77ac2ed0f97eabef1b2c80b11cdec17d4c0
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M t/loc_tools.pl

  Log Message:
  -----------
  t/loc_tools.pl: Check for unsupported locales

This commit causes us not to view unsupported locales as legitimate for
testing.  Core dumps occurred on some platforms without this.

It looks for a diagnostic that the next commit in this series will
generate.


  Commit: e3584473310db81dbec8ef16618335d042036a1e
      https://github.com/Perl/perl5/commit/e3584473310db81dbec8ef16618335d042036a1e
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M t/loc_tools.pl

  Log Message:
  -----------
  t/loc_tools.pl: White-space only


  Commit: 8ff394686973620ddd42815c70ab7e38d6ae8bb2
      https://github.com/Perl/perl5/commit/8ff394686973620ddd42815c70ab7e38d6ae8bb2
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M t/loc_tools.pl

  Log Message:
  -----------
  t/loc_tools.pl: Turn off warnings in a timely manner

It doesn't matter much, but some warnings might be output by doing the
'use locale' before turning off warnings.


  Commit: 7cd658bbb5a68df1d23f0ab4c4e762bd207aa685
      https://github.com/Perl/perl5/commit/7cd658bbb5a68df1d23f0ab4c4e762bd207aa685
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M embed.fnc
    M embed.h
    M locale.c
    M proto.h

  Log Message:
  -----------
  locale.c: querylocale return mortalized copy

It is too easy to forget to savepv() the return of these macros, leading
to hard-to-diagnose bugs.  Head those off at the pass by always making a
copy that gets freed by the system.


  Commit: 1a7107ebdb72c1bc33083ba95a1e59ade395e435
      https://github.com/Perl/perl5/commit/1a7107ebdb72c1bc33083ba95a1e59ade395e435
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M locale.c

  Log Message:
  -----------
  Initialize PL_numeric_name, PL_collation_name

Having these initialized to the C locale aoids some otherwise required
conditionals.


  Commit: f8229a1757e4092e98965a55a97198ddaa5d5113
      https://github.com/Perl/perl5/commit/f8229a1757e4092e98965a55a97198ddaa5d5113
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M locale.c

  Log Message:
  -----------
  locale.c: Silence some compiler warnings if no LC_ALL


  Commit: 4c338c5ceb70d92dd35fb1489e2105ac11305097
      https://github.com/Perl/perl5/commit/4c338c5ceb70d92dd35fb1489e2105ac11305097
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M locale.c

  Log Message:
  -----------
  locale.c: Don't change locale if already there

Changing the locale is cheap for some categories, but expensive for
others.  Changing LC_COLLATE is most expensive, requiring recalculation
of the collation transformation mapping.

This commit checks that we aren't already in the desired locale before
changing locales. and does nothing if no change is needed.


  Commit: 8294dfbb43e793db63bcbb930c92d529a4bad256
      https://github.com/Perl/perl5/commit/8294dfbb43e793db63bcbb930c92d529a4bad256
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M embed.fnc
    M embed.h
    M locale.c
    M proto.h

  Log Message:
  -----------
  locale.c: Separate out two Win fcns from a larger one

This makes the larger one easier to understand, and prepares for
possible independent calls to the two, which are potentially useful on
their own.


  Commit: b4234b11281f99174eb122e62f011dd51bdbbce6
      https://github.com/Perl/perl5/commit/b4234b11281f99174eb122e62f011dd51bdbbce6
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M embed.fnc
    M embed.h
    M ext/POSIX/POSIX.xs
    M ext/POSIX/lib/POSIX.pm
    M locale.c
    M proto.h

  Log Message:
  -----------
  Move code from POSIX.xs to locale.c

This avoids duplicated logic.


  Commit: 0258ff7048a7f0cb5abce5ece2455e44e34c79cc
      https://github.com/Perl/perl5/commit/0258ff7048a7f0cb5abce5ece2455e44e34c79cc
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M locale.c

  Log Message:
  -----------
  locale.c: Reorder cases in a switch

This moves handling the CODESET to the end, as future commits will make
its handling more complicated.  The cases are now ordered so the
simplest (based on the direction of future commits) are first


  Commit: c7701449ac01c4997c91f012cf5ef7bd72ae6461
      https://github.com/Perl/perl5/commit/c7701449ac01c4997c91f012cf5ef7bd72ae6461
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M locale.c

  Log Message:
  -----------
  locale.c: Make statics of repeated string constants

These strings are (or soon will be) used in multiple places; so have
just one definition for them.


  Commit: ff0a06606948fd73603268f225e5f139b6e9d22d
      https://github.com/Perl/perl5/commit/ff0a06606948fd73603268f225e5f139b6e9d22d
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M locale.c

  Log Message:
  -----------
  locale.c: Add two #defines

This makes sure that we handle having any variant of nl_langinfo() or
localeconv().


  Commit: 901cfe897aba52d26520ad7990ae95b98091bda5
      https://github.com/Perl/perl5/commit/901cfe897aba52d26520ad7990ae95b98091bda5
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M embed.fnc
    M embed.h
    M locale.c
    M proto.h

  Log Message:
  -----------
  locale.c: Return defaults for uncomputable langinfo items

Return the values from the C locale for nl_langinfo() items that aren't
computable on this platform.  If the platform has nl_langinfo(), then
all of them are computable, but if not, some can't be computed, and
others can be, but only if there are alternative methods available on
the platform.

As part of this commit, S_my_nl_langinfo() and S_save_to_buffer() are no
longer used when USE_LOCALE is not defined, so don't compile them.


  Commit: 5ad56922451adcbbc1c0e96812a13251b11b4a1e
      https://github.com/Perl/perl5/commit/5ad56922451adcbbc1c0e96812a13251b11b4a1e
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M locale.c

  Log Message:
  -----------
  locale.c: Rmv reimplementation of my_strftime()

Prior to this commit, there was a near duplicate copy of the code from
util.c that implements my_strftime().  This was done because the util.c
version zaps the wday field, which made it incompatible.

But it dawned on me that if the arbitrary date we use to do our
calculations were such that it was for a year in which the wday field
gets zapped to the value we want it to be, then the util.c version
automatically works.  This happens in years when January 1 falls on a
Sunday.


  Commit: d250a19691b9f9f2869f266ee9cf005c43df0bf5
      https://github.com/Perl/perl5/commit/d250a19691b9f9f2869f266ee9cf005c43df0bf5
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M locale.c

  Log Message:
  -----------
  locale.c: White-space only

Align with previous commit and properly indent some preprocessor
directives


  Commit: 72ada239ba4eb00c6e569cb3769ddbde15479960
      https://github.com/Perl/perl5/commit/72ada239ba4eb00c6e569cb3769ddbde15479960
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M embed.fnc
    M embed.h
    M locale.c
    M proto.h

  Log Message:
  -----------
  locale.c: Shorten static function name

The extra syllable(s) are unnecessary noise


  Commit: f68aa33c85ccdbd1a572c22f38f82633773f133f
      https://github.com/Perl/perl5/commit/f68aa33c85ccdbd1a572c22f38f82633773f133f
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M embed.fnc
    M locale.c
    M proto.h

  Log Message:
  -----------
  locale.c: Extend a static function

This will allow it to be used in situations where the buffer it controls
is single use, and we don't need to keep track of the size for future
calls.


  Commit: 9189df732c79e4cef07220a378e4d1b536a02d1e
      https://github.com/Perl/perl5/commit/9189df732c79e4cef07220a378e4d1b536a02d1e
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M locale.c

  Log Message:
  -----------
  locale.c: Use typedef to simplify

This allows some preprocessor conditionals to be removed


  Commit: 1a8737262a17370f9aebf70905cc3d6635e1b382
      https://github.com/Perl/perl5/commit/1a8737262a17370f9aebf70905cc3d6635e1b382
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M locale.c

  Log Message:
  -----------
  locale.c: Rmv redundant cBOOL()

strEQ and && already return booleans


  Commit: 2c235dd27cd282ec0f3e2133fc5ab3957d03b534
      https://github.com/Perl/perl5/commit/2c235dd27cd282ec0f3e2133fc5ab3957d03b534
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M locale.c

  Log Message:
  -----------
  locale.c: Fix currency symbol derivation

On platforms without nl_langinfo(), we derive the currency symbol from
localeconv().  The symbol must be tweaked to conform to nl_langinfo()
standards.  Prior to this commit, it guessed at how to tweak a rare
circumstance.  I found evidence this guess was wrong, so looked around,
and copied the way cygwin does it.

This also no longer returns just an empty string in certain cases.
nl_langinfo() itself doesn't, so conform to that.


  Commit: 37f1b9c38764ec6a13d1b6d6f04c51a034c2d5a4
      https://github.com/Perl/perl5/commit/37f1b9c38764ec6a13d1b6d6f04c51a034c2d5a4
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M locale.c

  Log Message:
  -----------
  locale.c: Don't add CP to Windows code page names

The actual name appears to be just the number for purposes of
nl_langinfo()-ish things.


  Commit: 9b8fe29275378995992199597099c9552604eeb8
      https://github.com/Perl/perl5/commit/9b8fe29275378995992199597099c9552604eeb8
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M embed.fnc
    M locale.c
    M proto.h

  Log Message:
  -----------
  locale.c: Don't ask a static fcn to be inlined

It's too complicated to really be inlined, and the compiler can figure
things out itself given it is a static function


  Commit: e7bb6f5683025a0517c7874f29c908a2830d42bb
      https://github.com/Perl/perl5/commit/e7bb6f5683025a0517c7874f29c908a2830d42bb
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M embed.fnc
    M proto.h

  Log Message:
  -----------
  S_save_to_buffer() allow ignoring return value

Future commits will want to use this, while discarding the return value.


  Commit: 39700f88dc7ed6dabf8ef66d14ca9f47192a55cf
      https://github.com/Perl/perl5/commit/39700f88dc7ed6dabf8ef66d14ca9f47192a55cf
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M embed.fnc
    M locale.c
    M proto.h

  Log Message:
  -----------
  locale.c: Rmv no longer used param from static fnc

Previous commits have gotten rid of this parameter to S_save_to_buffer


  Commit: 1a1aa07988145ab3bb4c4ca37c72d1d0496768c9
      https://github.com/Perl/perl5/commit/1a1aa07988145ab3bb4c4ca37c72d1d0496768c9
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M locale.c

  Log Message:
  -----------
  locale.c: Don't read off buffer end

In some configurations, under the exact set of input it would have been
possible to read past the buffer end.  This commit adds a conditional to
prevent that.


  Commit: f8cc04179ae1e194a61c1ad2acf3669389ee875f
      https://github.com/Perl/perl5/commit/f8cc04179ae1e194a61c1ad2acf3669389ee875f
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M locale.c

  Log Message:
  -----------
  locale.c: Fix Windows bug with broken localeconv()

localeconv() was broken on Windows until VS 2015.  As a workaround, this
was using my_snprintf() to find what the decimal point character is,
trying to avoid our workaround for localeconv(), which has a (slight)
chance of a race condition.

The problem is that my_snprintf() might not end up calling snprintf at
all; I didn't trace all possibilities in Windows.  So it doesn't make
for a reliable sentinel.

This commit now specifically uses libc snprintf(), and if it fails, drops
down to try localeconv().

It also changes things so that if localeconv() is not present at all or
usable on the platform, to use this snprintf method.


  Commit: 282866edc576664f4a55e7c504c0769e6669ef65
      https://github.com/Perl/perl5/commit/282866edc576664f4a55e7c504c0769e6669ef65
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M locale.c

  Log Message:
  -----------
  locale.c: Use a scratch buf; instead of reusing old

This is in preparation for the next commit


  Commit: 8ada8757abbdc349dada92d85761cc77f98e7875
      https://github.com/Perl/perl5/commit/8ada8757abbdc349dada92d85761cc77f98e7875
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M intrpvar.h
    M locale.c

  Log Message:
  -----------
  locale: make PL_langinfo_buf const *

The previous commit allows this change to be made.


  Commit: 91cbb4a4b821c4cdad46471161fb06a6f6d6dde5
      https://github.com/Perl/perl5/commit/91cbb4a4b821c4cdad46471161fb06a6f6d6dde5
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M embed.fnc
    M embed.h
    M proto.h

  Log Message:
  -----------
  embed.fnc: Also check for NL_LANGINFO_L

The preprocessor directives were only flooking for plain nl_langinfo().
It's quite unlikely that a platform will have the '_l' version without
also having the plain one.  But this makes sure.


  Commit: b096f0f7a7d17e51bd14be986da87c140302e0eb
      https://github.com/Perl/perl5/commit/b096f0f7a7d17e51bd14be986da87c140302e0eb
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M embed.fnc
    M embed.h
    M locale.c
    M proto.h

  Log Message:
  -----------
  locale.c: Make static fcn reentrant

This makes my_langinfo() reentrant by adding parameters specifying where
to store the result.

This prepares for future commits, and fixes some minor bugs for XS
writers, in that the claim was that the buffer in calling
Perl_langinfo() was safe from getting zapped until the next call to it
in the same thread.  It turns out there were cases where, because of
internal calls, the buffer did get zapped.


  Commit: 52d82eec7f61d90ace37a1ad4c88afc01d896761
      https://github.com/Perl/perl5/commit/52d82eec7f61d90ace37a1ad4c88afc01d896761
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M locale.c

  Log Message:
  -----------
  locale.c: langinfo: Use Windows fcn to find CODESET

There is a Windows function, available for quite a long time, that will
return the current code page.  Use this for the nl_langinfo() CODESET,
as that libc function isn't implemented on Windows.


  Commit: 9d2a9a5ade7d9a34111ed092d1b5435c4e121b78
      https://github.com/Perl/perl5/commit/9d2a9a5ade7d9a34111ed092d1b5435c4e121b78
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M embed.fnc
    M embed.h
    M locale.c
    M proto.h

  Log Message:
  -----------
  locale.c: Add static fcn to analyze locale name codeset

It determines if the name indicates it is UTF-8 or not.  There are
several variant spellings in use, and this hides that from the the
callers.

It won't be actually used until the next commit


  Commit: f3077a6520427eb40a0250a6c9a84571ff9b1cda
      https://github.com/Perl/perl5/commit/f3077a6520427eb40a0250a6c9a84571ff9b1cda
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M ext/I18N-Langinfo/Langinfo.pm
    M locale.c

  Log Message:
  -----------
  locale.c: Improve non-nl_langinfo() CODESET calc

Prior to this commit, on non-Windows platforms that don't have a
nl_langinfo() libc function, the code completely punted computation of
the CODESET item.  I have not been able to figure out how to do this,
even going to the locale definition files on disk (which may vary
anyway), but we can do a lot better than punting.

This commit adds three checks:

1) If the locale name is C or POSIX, we know the codeset

2) We can detect if a locale is UTF-8.  If it is, that is the codeset.
Many modern locales are of this ilk.

3) Failing that, some locales have the codeset appear in the name,
following a dot.

It isn't perfect, but it's a lot better than completely punting.


  Commit: c24bd3ed9671877106f159e98ce54abebd393ca0
      https://github.com/Perl/perl5/commit/c24bd3ed9671877106f159e98ce54abebd393ca0
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M embed.fnc
    M locale.c
    M proto.h

  Log Message:
  -----------
  Add toggle_locale() fcns

These are designed to temporarily switch the locale for a cateogry
around some operation that needs it to be different than the current
one.  They will be used in the next commit.

These will eventually replace the more unwieldy
_is_cur_LC_category_utf8() function, which toggles as a side effect


  Commit: a27ea2c51807bf5d715b963046e06c0417c4d6a8
      https://github.com/Perl/perl5/commit/a27ea2c51807bf5d715b963046e06c0417c4d6a8
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M embed.fnc
    M embed.h
    M locale.c
    M proto.h

  Log Message:
  -----------
  New signature for static fcn my_langinfo()

This commit changes the calling sequence for my_langinfo to add the
desired locale, and the locale category of the desired item.

This allows the function to be able to return the desired value for any
locale, avoiding some locale changes that would happen until this
commit, and hiding the need for locale changes from outside functions,
though a couple continue to do so to avoid potential multiple changes.


  Commit: d806a58f47148b49b4ef7ff771099b1e4e32bd0d
      https://github.com/Perl/perl5/commit/d806a58f47148b49b4ef7ff771099b1e4e32bd0d
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M locale.c

  Log Message:
  -----------
  locale.: Need CTYPE to match other category for nl_langinfo

nl_langinfo knows about various components of locales that are supposed
to be defined for every locale, such as a string for a Yes/No response
or the name of a month in a particular language.  These are associated
with various locale categories.  In the examples cited, the month names
are in the LC_TIME category, and the responses in the LC_MESSAGES one.
But (perhaps because these are text strings), some platforms require the
LC_CTYPE locale to be the same as the other locale.  cygwin is an
example.  Rather than try to figure out which platform require this, and
which do not, it is a simple matter to just LC_CTYPE at the same time as
the other category


  Commit: 9065729b6b5d12baf81916d3aaf60507fe4f890e
      https://github.com/Perl/perl5/commit/9065729b6b5d12baf81916d3aaf60507fe4f890e
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M embed.fnc
    M embed.h
    M locale.c
    M proto.h

  Log Message:
  -----------
  locale.c: Add is_locale_utf8()

Previous commits have added the infrastructure to be able to determine
if a locale is UTF-8.  This will prove useful, and this commit adds
a function to encapsulate this information, and uses it in a couple of
places, with more to come in future commits.

This uses as a final fallback, mbtowc(), supposed to be available in
C99.  Future commits will add heuristics when that function isn't
available or is known to be unreliable on a particular system.


  Commit: 4ceba7e42767bab802d5db14416177f6d145ad8f
      https://github.com/Perl/perl5/commit/4ceba7e42767bab802d5db14416177f6d145ad8f
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M embed.fnc
    M embed.h
    M locale.c
    M perl.h
    M proto.h
    M utf8.h

  Log Message:
  -----------
  locale.c: Add fcn for UTF8ness determination

get_locale_string_utf8ness_i() will determine if the string it is passed
in the locale it is passed is to be treated as UTF-8, or not.


  Commit: 1f0ba0b9e309240d281e42c212979d9291e081ec
      https://github.com/Perl/perl5/commit/1f0ba0b9e309240d281e42c212979d9291e081ec
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M embed.fnc
    M embed.h
    M ext/POSIX/POSIX.xs
    M locale.c
    M proto.h

  Log Message:
  -----------
  XXX perldelta Move POSIX::localeconv() logic to locale.c

The code currently in POSIX.xs is moved to locale.c, and reworked some
to fit in that scheme, and the logic for the workaround for the Windows
broken localeconv() is made more robust.

This is in preparation for the next commit which will use this logic
instead of (imperfectly) duplicating it.

This also creates Perl_localeconv() for direct XS calls of this
functionality.


  Commit: 40fe2d6fd9c9e398ea8f33f8f199704884f71c04
      https://github.com/Perl/perl5/commit/40fe2d6fd9c9e398ea8f33f8f199704884f71c04
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M locale.c

  Log Message:
  -----------
  locale.c: localeconv() unconditional NUMERIC toggle

It is possible to lockout changing the LC_NUMERIC locale.  This is done
in some printf cases where a recursive call could get the radix
character wrong.  But localeconv(), which could be called during this
recursion on some platforms, toggles the locale briefly, without
affecting the surrounding calls; so it can do the toggle
unconditionally.

The previous commit merely moved the functionality of localeconv() from
POSIX.xs to locale.c.  This commit expands upon that.


  Commit: cd977cb84ae687a506f708b1eceb6ed0e92cb33b
      https://github.com/Perl/perl5/commit/cd977cb84ae687a506f708b1eceb6ed0e92cb33b
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M embed.fnc
    M embed.h
    M locale.c
    M proto.h

  Log Message:
  -----------
  locale.c: Collapse duplicate logic into one instance

A previous commit move the logic for localeconv() into locale.c.  This
commit takes advantage of that to use it instead of repeating the logic.

Notably, this commit removes the inconsistent duplicate logic that had
been used to deal with the Windows broken localeconv() bug.


  Commit: 0f56a77e6f673ecf42056e9e605bf6080605ef05
      https://github.com/Perl/perl5/commit/0f56a77e6f673ecf42056e9e605bf6080605ef05
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M locale.c

  Log Message:
  -----------
  locale.c: Add branch prediction, comments


  Commit: 885e248b5af66524abb51d70dd880eae7a55ce5c
      https://github.com/Perl/perl5/commit/885e248b5af66524abb51d70dd880eae7a55ce5c
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M embed.fnc
    M embed.h
    M ext/POSIX/POSIX.xs
    M locale.c
    M proto.h

  Log Message:
  -----------
  XXXdelta Add my_strftime8()

This is like plain my_strftime(), but additionally returns an indication
of the UTF-8ness of the returned string


  Commit: 0f8030e955997fc6e06e936e8acdf422d8324695
      https://github.com/Perl/perl5/commit/0f8030e955997fc6e06e936e8acdf422d8324695
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M embed.fnc
    M embed.h
    M locale.c
    M proto.h

  Log Message:
  -----------
  locale.c: Add utf8ness return param to static fcn

my_langinfo_i() now will additionally return the UTF-8ness of the
returned string.


  Commit: 0c244c8a763d0a7a671adacc59a09daac3436c6f
      https://github.com/Perl/perl5/commit/0c244c8a763d0a7a671adacc59a09daac3436c6f
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M embed.fnc
    M ext/I18N-Langinfo/Langinfo.xs
    M locale.c
    M proto.h

  Log Message:
  -----------
  XXXdelta Add Perl_langinfo8()

This is like Perl_langinfo() but additionally returns information about
the UTF-8ness of the returned string.


  Commit: 5f7754ea61daf5535ca801b41a8b743cce77e0df
      https://github.com/Perl/perl5/commit/5f7754ea61daf5535ca801b41a8b743cce77e0df
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M lib/locale.t

  Log Message:
  -----------
  lib/locale.t: Use I18N::Langinfo, not POSIX::localeconv()

Now that Langinfo is ported to every box, it requires less work than
localeconv(), and offers more choices.  This commit changes to use it,
and for more info when debugging, gets some additional info from it,
while avoiding some calls when not debugging


  Commit: 2353c781edf95ead328555d5fe22a81c7db0d2c3
      https://github.com/Perl/perl5/commit/2353c781edf95ead328555d5fe22a81c7db0d2c3
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M lib/locale_threads.t

  Log Message:
  -----------
  locale_threads.t: Use I18N::Langinfo, not POSIX::localeconv()

The former is always present; the latter might not be


  Commit: a36a9cfefe146962192ea3d3e8a6fc897a189221
      https://github.com/Perl/perl5/commit/a36a9cfefe146962192ea3d3e8a6fc897a189221
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M locale.c

  Log Message:
  -----------
  locale.c: Add fallbacks if no mbtowc()

This add heuristics that work well for non-English locales to determine
if a locale is UTF-8 or not when mbtowc() isn't available.  It would be
a very rare compiler that didn't have that these days, but this covers
that case as best as I have been able to figure out.


  Commit: c5ec42cef2005e48df57061bfd7e5faaad4eb49d
      https://github.com/Perl/perl5/commit/c5ec42cef2005e48df57061bfd7e5faaad4eb49d
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M ext/I18N-Langinfo/t/Langinfo.t

  Log Message:
  -----------
  Revert "XXX Temporarily skip on Windows"

This should now be fixed by intervening commits


  Commit: de1330a132e213732cde8e1e7fedf2129ae8cf0b
      https://github.com/Perl/perl5/commit/de1330a132e213732cde8e1e7fedf2129ae8cf0b
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M perl.c

  Log Message:
  -----------
  perl.c: Rmv redundant initialization of PL_C_locale_obj

This is now initialized with the rest of the locale variables in
locale.c initialization.

Spotted by Tony Cook


  Commit: 334448ed0e46b6ec5e2c74f76291738d638485e2
      https://github.com/Perl/perl5/commit/334448ed0e46b6ec5e2c74f76291738d638485e2
  Author: Karl Williamson <khw@cpan.org>
  Date:   2022-08-11 (Thu, 11 Aug 2022)

  Changed paths:
    M locale.c

  Log Message:
  -----------
  XXX temp win32 debugging


Compare: https://github.com/Perl/perl5/compare/2d0e205decae...334448ed0e46



nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About