1
0
Fork 0
mirror of git://git.code.sf.net/p/cdesktopenv/code synced 2025-03-09 15:50:02 +00:00
cde/src/cmd/ksh93/tests/_common
Martijn Dekker db3a3d8fc0 tests/leaks.sh: redesign with a more robust testing algorithm
On modern operating systems, memory management is non-deterministic
(i.e. random, unpredictable) to varying degrees. This makes testing
for memory leaks a nightmare as the OS may decide to randomly grow
a process's memory allocation at any time for no apparent reason,
causing intermittent test failures that do not represent real
memory leaks. So far, the leaks test tried to cope with this by
using a large number of iterations plus a certain amount of bytes
of tolerance per iteration. This was inefficient and on some
systems still did not fully eliminate intermittent test failures.

This commit introduces a new testing algorithm that is designed to
cope with a large degree of unpredictability. Instead of a fixed
number of test iterations, it defines a maximum (16384), dividing
them in blocks of 128 iterations. It also defines a minimum number
of sequential "good" iteration blocks, counted if memory usage did
not increase from one block to the next. That minimum number is set
to 16. The theory is that if we can get 16 "good" iteration blocks
in a row, we can safely assume it's not a real memory leak, break
the loop, and consider the test succeeded. That "good" sequence is
allowed to occur at any point in the loop, creating a high built-in
tolerance for non-deterministic shenanigans. It also speeds up the
tests, as successful tests can bow out at 16 * 128 == 2048
iterations if they're lucky. If the OS decides to randomly grow the
memory heap, it may take more tries, but almost (?) certainly not
more than the maximum 16384 (128 blocks). If the counter reaches
that, then we assume a memory leak and throw a test failure.

We're also no longer testing with byte granularity in any case; the
randomness of memory management makes that pointless. All getmem()
function versions now return kibibytes (1024 bytes).

This should eliminate the need for workarounds such as initial
iterations to "steady the state" or a tolerance of a certain number
of bytes. I've experimentally determined the exact values
(max_iter, block_iter, min_good_blocks) that seem to work reliably
on all systems I've tested. They are easy to tweak if necessary.

To make all this manageable, this commit hides all the supporting
code in a triplet of aliases (TEST, DO, DONE) that, when used
correctly, create a grammatically robust shell code block: you can
add redirections, pipe into it, etc. as expected. This makes the
actual tests a great deal easier to read as well.

src/cmd/ksh93/tests/pty.sh:
- Implement new leaks testing framework as described and convert
  all the tests to it.
- Mark known leaks with a 'known' variable. Print non-fail warnings
  for all known leaks, but skip the tests by default. Test them
  only if DEBUG is exported. This is better than commenting them
  out as we will no longer be tempted to forget about these.
- Move the test for large command substitutions to subshell.sh --
  it's not in fact a leak test; instead, it checks that command
  substitutions don't lose data.

src/cmd/ksh93/tests/_common: err_exit():
- Since we're printing more warnings, clearly mark all test
  failures with 'FAIL:' to make them stand out.

src/cmd/ksh93/tests/shtests.
src/cmd/ksh93/tests/pty.sh:
- Special-case leaks.sh for counting tests; grep ^TEST.
- Special-case pty.sh as well while we're at it by grepping tst()
  calls. Remove all the dummy '# err_exit #' comments from pty.sh
  as they are now no longer used for counting the tests.
2021-12-28 17:47:29 +00:00

41 lines
1.7 KiB
Text

########################################################################
# #
# This file is part of the ksh 93u+m package #
# Copyright (c) 1982-2012 AT&T Intellectual Property #
# Copyright (c) 2021 Contributors to ksh 93u+m #
# <https://github.com/ksh93/ksh> #
# and is licensed under the #
# Eclipse Public License, Version 1.0 #
# #
# A copy of the License is available at #
# http://www.eclipse.org/org/documents/epl-v10.html #
# (with md5 checksum b35adb5213ca9657e911e9befb180842) #
# #
# Martijn Dekker <martijn@inlv.org> #
# #
########################################################################
_message()
{
print -r $'\t'"${Command}[$1]: ${@:2}" >&2
}
function err_exit
{
_message "$1" "FAIL:" "${@:2}"
let Errors+=1
}
alias err_exit='err_exit $LINENO' # inaccurate err_exit name kept for historical integrity :)
alias warning='_message "$LINENO" "warning:"'
Command=${0##*/}
integer Errors=0
if [[ -o xtrace ]]
then # PS4 is set and exported in shtests
typeset -F SECONDS # show fractional seconds in xtrace output
fi
if ! [[ -d $tmp && -w $tmp && $tmp == "$PWD" ]]
then print -r "$Command: \$tmp not set; run this from shtests. Aborting." >&2
exit 1
fi