arsenal package¶

Subpackages¶

Submodules¶

arsenal.alphabet module¶

class arsenal.alphabet.Alphabet(data=())[source]¶

Bases: object

Class for maintaining a perfect hash for a set of keys.

>>> a = Alphabet()
>>> [a[x] for x in 'abcd']
[0, 1, 2, 3]

>>> list(map(a.lookup, range(4)))
['a', 'b', 'c', 'd']

>>> a.freeze()
>>> a.add('z')
Traceback (most recent call last):
  ...
ValueError: Alphabet is frozen. Key "z" not found.

>>> print(a.plaintext())
a
b
c
d

>>> print(a)
Alphabet(size=4,frozen=True)

>>> list(a)
['a', 'b', 'c', 'd']

>>> a == Alphabet(['a', 'b', 'c', 'd'])
True

>>> a == Alphabet(['b', 'a', 'c', 'd'])
False

>>> a.map('aabc')
[0, 0, 1, 2]

add(k)¶

add_many(x)[source]¶

decode(i)¶

decode_many(x)¶

encode(k)¶

encode_many(seq)¶

freeze()[source]¶

imap(seq)[source]¶: Apply alphabet to sequence while filtering. By default, None is not emitted, so the Note that the output sequence may have fewer items.

items()[source]¶

classmethod load(filename)[source]¶

lookup(i)[source]¶

lookup_many(x)[source]¶

map(seq)[source]¶

plaintext()[source]¶: assumes keys are strings

save(filename)[source]¶

arsenal.assertions module¶

class arsenal.assertions.assert_throws(*expect)[source]¶

Bases: object

Contextmanager + Decorator for asserting that a certain exceptions (or no exception) will arise with it’s context.

As a context manager:

>>> with assert_throws(ZeroDivisionError):
...     1/0

>>> with assert_throws(None):
...     pass

>>> with assert_throws(None, ZeroDivisionError):
...     pass

>>> with assert_throws(ZeroDivisionError):
...     pass
Traceback (most recent call last):
    ...
AssertionError: did not raise required ZeroDivisionError. Got None instead.

>>> with assert_throws(AssertionError, ZeroDivisionError):
...     pass
Traceback (most recent call last):
    ...
AssertionError: did not raise required AssertionError or ZeroDivisionError. Got None instead.

arsenal.download module¶

arsenal.download.download(url, usecache=True, cached=None, cachedir='cache~/', cachedonly=False, **opts)[source]¶

Download (or cache) url to file. On success: return file name of stored contents. Upon failure: return None.

Will retry tries times with pause seconds between each attempt to download.

Download will timeout after timeout seconds.

If cachedonly is enabled, this function will not download anything. It will simply return the cached filename if it exists.

arsenal.download.robust_download(url, filename, tries=3, pause=0.1, timeout=30, verbose=True)[source]¶

Attempts tries times to download and write contents url to filename. Will timeout after timeout seconds.

returns None upon failure and filename on success.

arsenal.download.urlread(url)[source]¶

arsenal.download.wget(url, filename)[source]¶: Wraps call to wget to download url to filename.

arsenal.fsutils module¶

File system utilities

arsenal.fsutils.atomicwrite(filename, mode=438, verbose=False)[source]¶

Write to filename atomically, if for some reason an error occurs in this context the contents of the file prior to entering will not be lost.

Args:: filename: str; the name of the file mode: permissions with which to create the file

arsenal.fsutils.cd(d=None)[source]¶

arsenal.fsutils.clear_dir(d)[source]¶

arsenal.fsutils.directories(d, abspath=False)[source]¶: recursively list all directories.

arsenal.fsutils.ensure_dir(f, verbose=False)[source]¶: Ensure directories need to create a file exist.

arsenal.fsutils.files(d, abspath=False)[source]¶: Recursively list all files.

arsenal.fsutils.filesize(f)[source]¶

Uses du to compute human readable summary of filesize.

It’s a wrapper around

$ du -hs filename

arsenal.fsutils.filetype(f)[source]¶

arsenal.fsutils.find(d, filterfn=None, abspath=False, glob=None, regex=None, dirs=False)[source]¶

Recursively walks directory d yielding files which satisfy filterfn. Set option relpath to False to output absolute paths.

glob: shell glob filter function regex: regex filter function dirs: only search for directories matching filterfn

arsenal.fsutils.find_new_title(d, filename)[source]¶: If file filename exists in directory d, adds or changes the end of the file title until a name is found that doesn’t yet exist. Returns the new file name (without directory). For instance, if file “Image (01).jpg” exists, returns “Image (02).jpg”.

arsenal.fsutils.mkdir(d, verbose=False)[source]¶: Ensure directories need to create a file exist.

class arsenal.fsutils.preserve_cwd(f=None)[source]¶

Bases: object

context-manager which doubles as a decorator that preserve current working directory.

Usage example:

As a decorator:

>>> before = os.getcwd()
>>> @preserve_cwd
... def foo():
...     os.chdir('..')
>>> foo()
>>> before == os.getcwd()
True

As a context-manager:

>>> before = os.getcwd()
>>> with preserve_cwd():
...     os.chdir('..')
>>> before == os.getcwd()
True

arsenal.fsutils.secure_filename(filename)[source]¶

Pass it a filename and it will return a secure version of it. This filename can then safely be stored on a regular file system and passed to os.path.join(). The filename returned is an ASCII only string for maximum portability.

On windows system the function also makes sure that the file is not named after one of the special device files.

>>> secure_filename("My cool movie.mov")
'My_cool_movie.mov'
>>> secure_filename("../../../etc/passwd")
'etc_passwd'
>>> secure_filename(u'i contain cool \xfcml\xe4uts.txt')
'i_contain_cool_umlauts.txt'
>>> secure_filename(u'no brackets [ ] allowed either.txt')
'no_brackets___allowed_either.txt'

The function might return an empty filename. It’s your responsibility to ensure that the filename is unique and that you generate random filename if the function returned an empty one.

arsenal.humanreadable module¶

arsenal.humanreadable.datestr(then, now=None)[source]¶

Converts a (UTC) datetime object to a nice string representation.

>>> d = datetime(1970, 5, 1)
>>> datestr(d, now=d)
'0 microseconds ago'
>>> datestr(datetime(1970, 1, 1), now=d)
'Jan 01'
>>> datestr(datetime(1969, 1, 1), now=d)
'Jan 01, 1969'
>>> datestr(datetime(1970, 6, 1), now=d)
'Jun 01, 1970'
>>> datestr(None)
''

arsenal.humanreadable.htime(s, show_seconds=True)[source]¶: Given a number of seconds, returns a string attempting to represent it as shortly as possible. >>> htime(100000) ‘1d3h46m40s’ >>> htime(1) ‘1s’ >>> htime(10) ’10s’ >>> htime(1000) ‘16m40s’ >>> htime(10000) ‘2h46m40s’

arsenal.humanreadable.marquee(txt='', width=78, mark='*')[source]¶

Return the input string centered in a ‘marquee’.

>>> marquee('hello', width=50)
'********************* hello *********************'

arsenal.humanreadable.nth(n)[source]¶

Formats an ordinal. Doesn’t handle negative numbers.

>>> nth(1)
'1st'
>>> nth(0)
'0th'
>>> [nth(x) for x in [2, 3, 4, 5, 10, 11, 12, 13, 14]]
['2nd', '3rd', '4th', '5th', '10th', '11th', '12th', '13th', '14th']
>>> [nth(x) for x in [91, 92, 93, 94, 99, 100, 101, 102]]
['91st', '92nd', '93rd', '94th', '99th', '100th', '101st', '102nd']
>>> [nth(x) for x in [111, 112, 113, 114, 115]]
['111th', '112th', '113th', '114th', '115th']

arsenal.humanreadable.print_elapsed_time()[source]¶: register an exit hook which prints the start, finish, and elapsed times of a script.

arsenal.humanreadable.str2bool(s)[source]¶

Convert a string to a boolean value. The supported conversions are:

String Boolean value

“false” False

“true” True

“f” False

“t” True

“0” False

“1” True

“n” False

“y” True

“no” False

“yes” True

“off” False

“on” True

Strings are compared in a case-blind fashion.

Note: This function is not currently localizable.

Parameters:	s : str The string to convert to boolean
Return type:	bool
Returns:	the corresponding boolean value
Raises:	ValueError – unrecognized boolean string

arsenal.humanreadable.timetuple(s)[source]¶: htime(x) -> (days, hours, minutes, seconds)

arsenal.integerizer module¶

class arsenal.integerizer.AbstractIntegerizer[source]¶

Bases: object

add(k)¶: Call self as a function.

decode(i)¶

encode(k)¶: Call self as a function.

lookup(i)¶

class arsenal.integerizer.FeatureHashing(h, bits)[source]¶

Bases: arsenal.integerizer.AbstractIntegerizer

>>> h = FeatureHashing(jenkins32, 8)
>>> h([1, 2, 3])
[182, 76, 156]

class arsenal.integerizer.Integerizer(data=None)[source]¶

Bases: arsenal.integerizer.AbstractIntegerizer

Class for maintaining a perfect hash for a set of keys.

>>> a = Integerizer()
>>> a(list('abcd'))
[0, 1, 2, 3]

>>> a(list('ace'))
[0, 2, 4]

>>> a[list(range(4))]
['a', 'b', 'c', 'd']

>>> a.freeze()
>>> a.add('z')
Traceback (most recent call last):
  ...
ValueError: Alphabet is frozen. Key "z" not found.

>>> list(a)
['a', 'b', 'c', 'd', 'e']

freeze()[source]¶

items()[source]¶

keys()[source]¶

arsenal.integerizer.jenkins32(a)[source]¶

arsenal.iterview module¶

arsenal.iterview.iterview(x: Iterable, msg: Prefix message = None, length: length hint; required when len(x) fails. = None, width: max width of progress bar = 78, show: do not show progress bar if False = True, transient=False)[source]¶

arsenal.misc module¶

arsenal.misc.browser(html)[source]¶

Display html in the default web browser without creating a temp file.

Instantiates a trivial http server and calls webbrowser.open with a URL to retrieve html from that server.

arsenal.misc.ctx_redirect_io(f=None)[source]¶

Usage example:

>>> with ctx_redirect_io() as io_target:
...    print('how is this for io?')
>>> io_target.getvalue()
'how is this for io?\n'

class arsenal.misc.ddict(f)[source]¶

Bases: dict

Variation on collections.defaultdict which allows default value callback to inspect missing key.

arsenal.misc.deprecated(use_instead=None, msg=None)[source]¶: This is a decorator which can be used to mark functions as deprecated. It will result in a warning being emitted when the function is used.

arsenal.misc.edit_with_editor(s=None)[source]¶

Open os.environ[‘EDITOR’] and load in text s.

Returns the text typed in the editor, after running strip().

arsenal.misc.editor(s=None)¶

Open os.environ[‘EDITOR’] and load in text s.

Returns the text typed in the editor, after running strip().

arsenal.misc.force(g)[source]¶: force evaluation of generator g.

arsenal.misc.highlighter(p, flags=0)[source]¶

arsenal.misc.ignore_error(color='red')[source]¶

arsenal.misc.open_diff(a, b, cmd='meld')[source]¶: View diff of string representations in dedicated diff program.

arsenal.misc.pager(s, cmd='less')[source]¶: Use the pager passed in and send string s through it.

arsenal.misc.piped()[source]¶: Returns piped input via stdin, else None.

arsenal.misc.redirect_io(f=None)¶

Usage example:

>>> with ctx_redirect_io() as io_target:
...    print('how is this for io?')
>>> io_target.getvalue()
'how is this for io?\n'

arsenal.nb module¶

Utilities for working with Jupyter notebooks.

arsenal.nb.psource(*functions)[source]¶

Print the source code for the given function(s).

Based on https://github.com/aimacode/aima-python/blob/master/notebook.py

arsenal.passwords module¶

arsenal.passwords.password(service, user)[source]¶

arsenal.passwords.set_password(service, user, pw=None)[source]¶

arsenal.profiling module¶

arsenal.profiling.main()[source]¶

arsenal.profiling.profile_viz(cmd, global_dict=None, local_dict=None, img='profile.png', out='profile.tmp', noctx=False)[source]¶: Run gprof2dot on the output for profiling.

arsenal.profiling.profiler(use='cprofile', filename='out.prof')[source]¶

arsenal.robust module¶

exception arsenal.robust.Timeout[source]¶: Bases: Exception

arsenal.robust.retry(tries=2, pause=None, suppress=(<class 'Exception'>, ), allow=(<class 'NameError'>, <class 'NotImplementedError'>))[source]¶

arsenal.robust.retry_apply(fn, args, kwargs=None, tries=2, pause=None, suppress=(<class 'Exception'>, ), allow=(<class 'NameError'>, <class 'NotImplementedError'>))[source]¶

Attempt to call fn up to tries times with the args as arguments

suppress: exceptions to be ignored up to the last retry-attempt. allow: exceptions which will not be suppressed; this helps avoid

stupid mistakes such as NameErrors and NotImplementedErrors.

arsenal.robust.timelimit(seconds)[source]¶

A decorator to limit a function to timeout seconds, raising Timeout. if it takes longer.

>>> def meaningoflife():
...     sleep(.2)
...     return 42
>>>
>>> timelimit(.1)(meaningoflife)()
Traceback (most recent call last):
    ...
Timeout: Call took longer than 0.1 seconds.
>>> timelimit(1)(meaningoflife)()
42

>>> with timelimit(.2):
...     sleep(1)
Traceback (most recent call last):
    ...
Timeout: Call took longer than 0.2 seconds.

>>> with timelimit(.2):
...     sleep(.1)
...     print('finished')
finished

arsenal.sysutil module¶

arsenal.sysutil.memory_usage()[source]¶: Return the memory usage of this process in MB.

arsenal.terminal module¶

arsenal.terminal.ansi(color=None, light=None, bg=3)[source]¶

arsenal.terminal.ansi2html(x)[source]¶

class arsenal.terminal.arrow[source]¶

Bases: object

class Long[source]¶

Bases: object

l = '⟸'¶

left = '⟸'¶

leftright = '⟺'¶

lr = '⟺'¶

r = '⟹'¶

right = '⟹'¶

d = '↓'¶

dl = '↙'¶

down = '↓'¶

dr = ('↘',)¶

l = '←'¶

left = '←'¶

class long[source]¶

Bases: object

l = '⟵'¶

left = '⟵'¶

leftright = '⟷'¶

lr = '⟷'¶

r = '⟶'¶

right = '⟶'¶

r = '→'¶

right = '→'¶

class squiggle[source]¶

Bases: object

r = '⟿'¶

right = '⟿'¶

u = '↑'¶

ul = '↖'¶

up = '↑'¶

ur = ('↗',)¶

class arsenal.terminal.bb[source]¶

Bases: object

A = '𝔸'¶

B = '𝔹'¶

C = 'ℂ'¶

D = '𝔻'¶

E = '𝔼'¶

F = '𝔽'¶

G = '𝔾'¶

H = 'ℍ'¶

I = '𝕀'¶

J = '𝕁'¶

K = '𝕂'¶

L = '𝕃'¶

M = '𝕄'¶

N = 'ℕ'¶

O = '𝕆'¶

P = 'ℙ'¶

Q = 'ℚ'¶

R = 'ℝ'¶

S = '𝕊'¶

T = '𝕋'¶

U = '𝕌'¶

V = '𝕍'¶

W = '𝕎'¶

X = '𝕏'¶

Y = '𝕐'¶

Z = 'ℤ'¶

class arsenal.terminal.colors[source]¶

Bases: object

ansi2html()¶

class arrow¶

Bases: object

class Long¶

Bases: object

l = '⟸'¶

left = '⟸'¶

leftright = '⟺'¶

lr = '⟺'¶

r = '⟹'¶

right = '⟹'¶

d = '↓'¶

dl = '↙'¶

down = '↓'¶

dr = ('↘',)¶

l = '←'¶

left = '←'¶

class long¶

Bases: object

l = '⟵'¶

left = '⟵'¶

leftright = '⟷'¶

lr = '⟷'¶

r = '⟶'¶

right = '⟶'¶

r = '→'¶

right = '→'¶

class squiggle¶

Bases: object

r = '⟿'¶

right = '⟿'¶

u = '↑'¶

ul = '↖'¶

up = '↑'¶

ur = ('↗',)¶

bad = '\x1b[1;31mbad\x1b[0m'¶

class bg[source]¶

Bases: object

black = '\x1b[0;40m%s\x1b[0m'¶

blue = '\x1b[0;44m%s\x1b[0m'¶

cyan = '\x1b[0;46m%s\x1b[0m'¶

green = '\x1b[0;42m%s\x1b[0m'¶

magenta = '\x1b[0;45m%s\x1b[0m'¶

orange = '\x1b[48;2;255;165;0m%s\x1b[0m'¶

red = '\x1b[0;41m%s\x1b[0m'¶

static rgb(r, g, b)[source]¶

white = '\x1b[0;47m%s\x1b[0m'¶

yellow = '\x1b[0;43m%s\x1b[0m'¶

black = '\x1b[0;30m%s\x1b[0m'¶

blue = '\x1b[0;34m%s\x1b[0m'¶

bold = '\x1b[1m%s\x1b[0m'¶

check = '\x1b[0;32m✔\x1b[0m'¶

cyan = '\x1b[0;36m%s\x1b[0m'¶

class dark[source]¶

Bases: object

black = '\x1b[2;30m%s\x1b[0m'¶

blue = '\x1b[2;34m%s\x1b[0m'¶

cyan = '\x1b[2;36m%s\x1b[0m'¶

green = '\x1b[2;32m%s\x1b[0m'¶

magenta = '\x1b[2;35m%s\x1b[0m'¶

red = '\x1b[2;31m%s\x1b[0m'¶

white = '\x1b[2;37m%s\x1b[0m'¶

yellow = '\x1b[2;33m%s\x1b[0m'¶

fail = '\x1b[1;31mfail\x1b[0m'¶

green = '\x1b[0;32m%s\x1b[0m'¶

italic = '\x1b[3m%s\x1b[0m'¶

leftarrow = '←'¶

class light[source]¶

Bases: object

black = '\x1b[1;30m%s\x1b[0m'¶

blue = '\x1b[1;34m%s\x1b[0m'¶

cyan = '\x1b[1;36m%s\x1b[0m'¶

green = '\x1b[1;32m%s\x1b[0m'¶

magenta = '\x1b[1;35m%s\x1b[0m'¶

red = '\x1b[1;31m%s\x1b[0m'¶

white = '\x1b[1;37m%s\x1b[0m'¶

yellow = '\x1b[1;33m%s\x1b[0m'¶

static line(n)[source]¶

magenta = '\x1b[0;35m%s\x1b[0m'¶

mark()¶

static mark_(x)[source]¶

normal = '\x1b[0m%s\x1b[0m'¶

ok = '\x1b[0;32mok\x1b[0m'¶

orange = '\x1b[38;2;255;165;0m%s\x1b[0m'¶

poop = '💩'¶

red = '\x1b[0;31m%s\x1b[0m'¶

render(**kwargs)¶: Render colorful string using ‘reset’ to mean ‘pop the color stack’ rather than go directly ‘normal’ color.

class rendering(y, debug=False)¶: Bases: object

reset = '\x1b[0m'¶

static rgb(r, g, b)[source]¶

rightarrow = '→'¶

strike = '\x1b[9m%s\x1b[0m'¶

static thick_line(n)[source]¶

thumbs_down = '👎'¶

thumbs_up = '👍'¶

timeout = '⌛'¶

underline = '\x1b[4m%s\x1b[0m'¶

warn = '\x1b[0;33mwarn\x1b[0m'¶

white = '\x1b[0;37m%s\x1b[0m'¶

xmark = '\x1b[2;31m✘\x1b[0m'¶

yellow = '\x1b[0;33m%s\x1b[0m'¶

arsenal.terminal.colorstring(s, c)[source]¶

arsenal.terminal.complete_filenames(text, line, begidx, endidx)[source]¶: Util for filename completion.

arsenal.terminal.console_width(minimum=None, default=80)[source]¶: Return width of available window area.

arsenal.terminal.mark(x)[source]¶

arsenal.terminal.marquee(msg='')[source]¶

arsenal.terminal.overline(xs)[source]¶

arsenal.terminal.render(y, **kwargs)[source]¶: Render colorful string using ‘reset’ to mean ‘pop the color stack’ rather than go directly ‘normal’ color.

class arsenal.terminal.rendering(y, debug=False)[source]¶: Bases: object

arsenal.terminal.subscript(x)[source]¶

arsenal.terminal.superscript(x)[source]¶

arsenal.terminal.tests()[source]¶

arsenal.timer module¶

class arsenal.timer.Benchmark(title)[source]¶

Bases: object

compare(statistic=<sphinx.ext.autodoc.importer._MockObject object>)[source]¶

items()[source]¶

keys()[source]¶

plot_feature(feature, timecol='timer', ax=None, **kw)[source]¶

plot_survival(*args, **kwargs)[source]¶: Show the probability each algorithm is still running.

run(methods, reps)[source]¶

values()[source]¶

class arsenal.timer.Timer(name=None)[source]¶

Bases: object

>>> from time import sleep
>>> a = Timer('A')
>>> b = Timer('B')
>>> with a:
...     sleep(0.5)
>>> with b:
...     sleep(1)
>>> a.compare(b)          #doctest:+SKIP
A is 2.0018x faster

bucket_filter(feature_to_bucket, bucket_filter)[source]¶

compare(other, attr='mean', verbose=True)[source]¶

dataframe(timecol='timer')[source]¶

filter(f, name=None)[source]¶

mean¶

median¶

plot_feature(feature, timecol='timer', ax=None, label=None, scatter=False, show_curve=False, **kw)[source]¶

plot_survival(ax=None)[source]¶

std¶

total¶

trim_slow(feature_to_bucket, threshold)[source]¶

arsenal.timer.main()[source]¶

arsenal.timer.timeit(name, fmt='{name} ({htime})', header=None)[source]¶: Context Manager which prints the time it took to run code block.

arsenal.timer.timers(title=None)[source]¶

arsenal package¶

Subpackages¶

Submodules¶

arsenal.alphabet module¶

arsenal.assertions module¶

arsenal.download module¶

arsenal.fsutils module¶

arsenal.humanreadable module¶

arsenal.integerizer module¶

arsenal.iterview module¶

arsenal.misc module¶

arsenal.nb module¶

arsenal.passwords module¶

arsenal.profiling module¶

arsenal.robust module¶

arsenal.sysutil module¶

arsenal.terminal module¶

arsenal.timer module¶

Module contents¶

arsenal

Navigation

Related Topics

String	Boolean value
“false”	False
“true”	True
“f”	False
“t”	True
“t”	True
“0”	False
“1”	True
“1”	True
“n”	False
“y”	True
“y”	True
“no”	False
“yes”	True
“yes”	True
“off”	False
“on”	True
“on”	True