Scipy Lecture Notes Lecture Notes - [PDF Document] (2024)

Python
MatplotlibSciKits Numpy
SciPy
IPython
IP[y]:
Cython
2017EDITION
Edited byGal VaroquauxEmmanuelle GouillartOlaf Vahtras
ScipyLecture Notes
www.scipy-lectures.org
Gal Varoquaux Emmanuelle Gouil lart Olav VahtrasChristopherBurns Adrian Chauve Robert Cimrman Christophe Combelles
Pierre de Buyl Ralf Gommers Andr Espaze ZbigniewJdrzejewski-Szmek Valentin Haenel Gert-Ludwig Ingold FabianPedregosa Didrik Pinte
Nicolas P. Rougier Pauli Virtanen
an d man y ot her s . . .
Contents
I Getting started with Python for science 2
1 Python scientific computing ecosystem 41.1 Why Python? . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . 41.2 The Scientific Python ecosystem . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 61.3 Before starting: Installing a working environment . . . .. . . . . . . . . . . . . . . . . . . . . . . . 81.4 The workflow:interactive environments and text editors . . . . . . . . . . . . .. . . . . . . . . . . 8
2 The Python language 122.1 First steps . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . 132.2 Basic types . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .142.3 Control Flow . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . 212.4 Definingfunctions . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . 252.5 Reusing code: scripts andmodules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . 302.6 Input and Output . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .382.7 Standard Library . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . 392.8 Exceptionhandling in Python . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . 432.9 Object-oriented programming(OOP) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . 46
3 NumPy: creating and manipulating numerical data 473.1 TheNumPy array object . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 473.2 Numerical operationson arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . 593.3 More elaborate arrays . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 723.4 Advanced operations . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . 763.5 Someexercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 803.6 Full code examples .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . 85
4 Matplotlib: plotting 964.1 Introduction . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . 964.2 Simple plot . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 974.3Figures, Subplots, Axes and Ticks . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . 1054.4 Other Types ofPlots: examples and exercises . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . 1074.5 Beyond this tutorial . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . 1134.6 Quick references . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1154.7Full code examples . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . 117
5 Scipy : high-level scientific computing 1835.1 Fileinput/output: scipy.io . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . 1845.2 Special functions:scipy.special . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . 1855.3 Linear algebra operations: scipy.linalg. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .185
i
5.4 Interpolation: scipy.interpolate . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . 1875.5 Optimization andfit: scipy.optimize . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . 1875.6 Statistics and random numbers:scipy.stats . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 1935.7 Numerical integration: scipy.integrate . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . 1955.8 Fast Fouriertransforms: scipy.fftpack . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . 1975.9 Signal processing: scipy.signal . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 2005.10 Image manipulation: scipy.ndimage . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . 2025.11 Summaryexercises on scientific computing . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . 2075.12 Full code examples for thescipy chapter . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . 220
6 Getting help and finding documentation 258
II Advanced topics 261
7 Advanced Python Constructs 2637.1 Iterators, generatorexpressions and generators . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . 2647.2 Decorators . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 2687.3 Context managers . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . 276
8 Advanced NumPy 2808.1 Life of ndarray . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . 2818.2 Universal functions . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2948.3Interoperability features . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . 3038.4 Array siblings:chararray, maskedarray, matrix . . . . . . . . . . . . . . . . . .. . . . . . . . . . 3068.5 Summary . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 3098.6 Contributing to NumPy/Scipy . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . 309
9 Debugging code 3139.1 Avoiding bugs . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . 3149.2 Debugging workflow . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . 3169.3Using the Python debugger . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . 3179.4 Debuggingsegmentation faults using gdb . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . 322
10 Optimizing code 32410.1 Optimization workflow . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 32510.2 Profiling Python code . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . 32510.3Making code go faster . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . 32710.4 Writing fasternumerical code . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . 329
11 Sparse Matrices in SciPy 33111.1 Introduction . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . 33111.2 Storage Schemes . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33311.3 Linear System Solvers . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . 34511.4 OtherInteresting Packages . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . 350
12 Image manipulation and processing using Numpy and Scipy35112.1 Opening and writing to image files . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . 35212.2Displaying images . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . 35412.3 Basicmanipulations . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . 35512.4 Image filtering . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . 35812.5 Feature extraction . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . 36312.6 Measuring objects properties:ndimage.measurements . . . . . . . . . . . . . . . . . . . . . . .. 36512.7 Full code examples . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . 37012.8Examples for the image processing chapter . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . 370
13 Mathematical optimization: finding minima of functions39613.1 Knowing your problem . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . 39713.2 A reviewof the different optimizers . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . 39913.3 Full code examples . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . 40513.4 Examples for the mathematicaloptimization chapter . . . . . . . . . . . . . . . . . . . . . . .. . . 405
ii
13.5 Practical guide to optimization with scipy . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . 43113.6 Specialcase: non-linear least-squares . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . 43313.7 Optimization withconstraints . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . 43513.8 Full code examples . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . 43613.9 Examples for the mathematical optimizationchapter . . . . . . . . . . . . . . . . . . . . . . . . . . 436
14 Interfacing with C 43714.1 Introduction . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . 43714.2 Python-C-Api . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .43814.3 Ctypes . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . 44314.4SWIG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . 44714.5 Cython . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . 45114.6 Summary . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . 45614.7 Further Reading and References . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 45614.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 456
III Packages and applications 458
15 Statistics in Python 46015.1 Data representation andinteraction . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . 46115.2 Hypothesis testing: comparing two groups .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .46615.3 Linear models, multiple factors, and analysis of variance .. . . . . . . . . . . . . . . . . . . . . . . 46815.4 Morevisualization: seaborn for statistical exploration . . . . . . . .. . . . . . . . . . . . . . . . . . 47315.5 Testing forinteractions . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . 47715.6 Full code for the figures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . 47815.7 Solutions to this chapters exercises .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 499
16 Sympy : Symbolic Mathematics in Python 50216.1 First Stepswith SymPy . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . 50316.2 Algebraic manipulations .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . 50416.3 Calculus . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 50516.4 Equation solving . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50716.5Linear Algebra . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . 508
17 Scikit-image: image processing 51017.1 Introduction andconcepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . 51117.2 Input/output, data types andcolorspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . 51217.3 Image preprocessing / enhancement . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . 51417.4Image segmentation . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . 51817.5 Measuringregions properties . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . 52017.6 Data visualization andinteraction . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . 52117.7 Feature extraction for computer vision. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 52217.8 Full code examples . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . 52317.9Examples for the scikit-image chapter . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 523
18 Traits: building interactive dialogs 53518.1 Introduction . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . 53618.2 Example . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . 53718.3 What are Traits . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .537
19 3D plotting with Mayavi 55419.1 Mlab: the scripting interface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . 55519.2 Interactive work . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .56119.3 Slicing and dicing data: sources, modules and filters . . .. . . . . . . . . . . . . . . . . . . . . . . . 56219.4 Animatingthe data . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . 56519.5 Making interactivedialogs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . 56619.6 Putting it together . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . 568
20 scikit-learn: machine learning in Python 569
iii
20.1 Introduction: problem settings . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . 57020.2 Basicprinciples of machine learning with scikit-learn . . . . . . . . .. . . . . . . . . . . . . . . . . 57420.3 Supervised Learning:Classification of Handwritten Digits . . . . . . . . . . . . . . .. . . . . . . . 57920.4 Supervised Learning: Regression of HousingData . . . . . . . . . . . . . . . . . . . . . . . . . . . .58320.5 Measuring prediction performance . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . 58620.6Unsupervised Learning: Dimensionality Reduction and Visualization .. . . . . . . . . . . . . . . 59120.7 The eigenfaces example:chaining PCA and SVMs . . . . . . . . . . . . . . . . . . . . . . .. . . . . 59420.8 Parameter selection, Validation, and Testing . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60020.9Examples for the scikit-learn chapter . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . 607
Index 649
iv
Scipy lecture notes, Edition 2017.1
Contents 1
Part I
Getting started with Python for science
2
Scipy lecture notes, Edition 2017.1
This part of the Scipy lecture notes is a self-containedintroduction to everything that is needed to use Pythonfor science,from the language itself, to numerical computing or plotting.
3
CHAPTER1Python scientific computing ecosystem
Authors: Fernando Perez, Emmanuelle Gouillart, Gal Varoquaux,Valentin Haenel
1.1 Why Python?
1.1.1 The scientists needs
Get data (simulation, experiment control),
Manipulate and process data,
Visualize results, quickly to understand, but also with highquality figures, for reports or publications.
1.1.2 Pythons strengths
Batteries included Rich collection of already existing bricks ofclassic numerical methods, plotting ordata processing tools. Wedont want to re-program the plotting of a curve, a Fouriertransform or a fittingalgorithm. Dont reinvent the wheel!
Easy to learn Most scientists are not payed as programmers,neither have they been trained so. Theyneed to be able to draw acurve, smooth a signal, do a Fourier transform in a fewminutes.
Easy communication To keep code alive within a lab or a companyit should be as readable as a bookby collaborators, students, ormaybe customers. Python syntax is simple, avoiding strange symbolsorlengthy routine specifications that would divert the reader frommathematical or scientific understand-ing of the code.
Efficient code Python numerical modules are computationallyefficient. But needless to say that a veryfast code becomes uselessif too much time is spent writing it. Python aims for quickdevelopment timesand quick execution times.
4
Scipy lecture notes, Edition 2017.1
Universal Python is a language used for many different problems.Learning Python avoids learning anew software for each newproblem.
1.1.3 How does Python compare to other solutions?
Compiled languages: C, C++, Fortran. . .
Pros
Very fast. For heavy computations, its difficult to outperformthese languages.
Cons
Painful usage: no interactivity during development, mandatorycompilation steps, ver-bose syntax, manual memory management. Theseare difficult languages for non pro-grammers.
Matlab scripting language
Pros
Very rich collection of libraries with numerous algorithms, formany different domains.Fast execution because these libraries areoften written in a compiled language.
Pleasant development environment: comprehensive and help,integrated editor, etc.
Commercial support is available.
Cons
Base language is quite poor and can become restrictive foradvanced users.
Not free.
Julia
Pros
Fast code, yet interactive and simple.
Easily connects to Python or C.
Cons
Ecosystem limited to numerical computing.
Still young.
Other scripting languages: Scilab, Octave, R, IDL, etc.
Pros
Open-source, free, or at least cheaper than Matlab.
Some features can be very advanced (statistics in R, etc.)
Cons
Fewer available algorithms than in Matlab, and the language isnot more advanced.
Some software are dedicated to one domain. Ex: Gnuplot to drawcurves. These pro-grams are very powerful, but they are restrictedto a single type of usage, such as plotting.
1.1. Why Python? 5
Scipy lecture notes, Edition 2017.1
Python
Pros
Very rich scientific computing libraries
Well thought out language, allowing to write very readable andwell structured code: wecode what we think.
Many libraries beyond scientific computing (web server, serialport access, etc.)
Free and open-source software, widely spread, with a vibrantcommunity.
A variety of powerful environments to work in, such as IPython,Spyder, Jupyter note-books, Pycharm
Cons
Not all the algorithms that can be found in more specializedsoftware or toolboxes.
1.2 The Scientific Python ecosystem
Unlike Matlab, or R, Python does not come with a pre-bundled setof modules for scientific computing. Beloware the basic buildingblocks that can be combined to obtain a scientific computingenvironment:
Python, a generic and modern computing language
The language: flow control, data types (string, int), datacollections (lists, dictionaries), etc.
Modules of the standard library: string processing, filemanagement, simple network protocols.
A large number of specialized modules or applications written inPython: web framework, etc. . . . andscientific computing.
Development tools (automatic testing, documentationgeneration)
See also:
chapter on Python language
Core numeric libraries
Numpy: numerical computing with powerful numerical arraysobjects, and routines to manipulatethem. http://www.numpy.org/
See also:
1.2. The Scientific Python ecosystem 6
http://ipython.readthedocs.io/en/stable/https://pythonhosted.org/spyderhttp://jupyter.org/http://jupyter.org/https://www.jetbrains.com/pycharmhttp://www.numpy.org/
Scipy lecture notes, Edition 2017.1
chapter on numpy
Scipy : high-level numerical routines. Optimization, regression,interpolation, etc http://www.scipy.org/
See also:
chapter on scipy
Matplotlib : 2-D visualization, publication-ready plotshttp://matplotlib.org/
See also:
chapter on matplotlib
Advanced interactive environments:
IPython, an advanced Python console http://ipython.org/
Jupyter, notebooks in the browser http://jupyter.org/
Domain-specific packages,
Mayavi for 3-D visualization
pandas, statsmodels, seaborn for statistics
1.2. The Scientific Python ecosystem 7
http://www.scipy.org/http://matplotlib.org/http://ipython.org/http://jupyter.org/
Scipy lecture notes, Edition 2017.1
sympy for symbolic computing
scikit-image for image processing
scikit-learn for machine learning
and much more packages not documented in the scipy lectures.
See also:
chapters on advanced topics
chapters on packages and applications
1.3 Before starting: Installing a working environment
Python comes in many flavors, and there are many ways to installit. However, we recommend to install ascientific-computingdistribution, that comes readily with optimized versions ofscientific modules.
Under Linux
If you have a recent distribution, most of the tools areprobably packaged, and it is recommended to use yourpackagemanager.
Other systems
There are several fully-featured Scientific Pythondistributions:
Anaconda
EPD
WinPython
Python 3 or Python 2?
In 2008, Python 3 was released. It is a major evolution of thelanguage that made a few changes. Some oldscientific code does notyet run under Python 3. However, this is infrequent and Python 3comes with manybenefits. We advise that you install Python 3.
1.4 The workflow: interactive environments and text editors
Interactive work to test and understand algorithms: In thissection, we describe a workflow combining inter-active work andconsolidation.
Python is a general-purpose language. As such, there is not oneblessed environment to work in, and not onlyone way of using it.Although this makes it harder for beginners to find their way, itmakes it possible for Pythonto be used for programs, in webservers, or embedded devices.
1.4.1 Interactive work
We recommend an interactive work with the IPython console, orits offspring, the Jupyter notebook. They arehandy to explore andunderstand algorithms.
Under the notebook
To execute code, press shift enter
1.3. Before starting: Installing a working environment 8
https://www.continuum.io/downloadshttps://store.enthought.com/downloadshttps://winpython.github.iohttp://ipython.orghttp://jupyter.readthedocs.io/en/latest/content-quickstart.html
Scipy lecture notes, Edition 2017.1
Start ipython:
In [1]: print('Hello world')Hello world
Getting help by using the ? operator after an object:
In [2]: print?Type: builtin_function_or_methodBase Class: StringForm: Namespace: Python builtinDocstring:
print(value, ..., sep=' ', end='\n', file=sys.stdout)
Prints the values to a stream, or to sys.stdout bydefault.Optional keyword arguments:file: a file-like object(stream); defaults to the current sys.stdout.sep: string insertedbetween values, default a space.end: string appended after the lastvalue, default a newline.
See also:
IPython user manual:http://ipython.org/ipython-doc/dev/index.html
Jupyter Notebook QuickStart:http://jupyter.readthedocs.io/en/latest/content-quickstart.html
1.4.2 Elaboration of the work in an editor
As you move forward, it will be important to not only workinteractively, but also to create and reuse Pythonfiles. For this,a powerful code editor will get you far. Here are several goodeasy-to-use editors:
Spyder: integrates an IPython console, a debugger, a profiler. ..
PyCharm: integrates an IPython console, notebooks, a debugger. .. (freely available, but commercial)
Atom
Some of these are shipped by the various scientific Pythondistributions, and you can find them in the menus.
As an exercise, create a file my_file.py in a code editor, andadd the following lines:
s = 'Hello world'print(s)
Now, you can run it in IPython console or a notebook and explorethe resulting variables:
In [1]: %run my_file.pyHello world
In [2]: sOut[2]: 'Hello world'
In [3]: %whosVariable TypeData/Info----------------------------s str Hello world
From a script to functions
While it is tempting to work only with scripts, that is a filefull of instructions following each other, do planto progressivelyevolve the script to a set of functions:
1.4. The workflow: interactive environments and text editors9
http://ipython.org/ipython-doc/dev/index.htmlhttp://jupyter.readthedocs.io/en/latest/content-quickstart.htmlhttps://pythonhosted.org/spyder/https://www.jetbrains.com/pycharmhttps://atom.io
Scipy lecture notes, Edition 2017.1
A script is not reusable, functions are.
Thinking in terms of functions helps breaking the problem insmall blocks.
1.4.3 IPython and Jupyter Tips and Tricks
The user manuals contain a wealth of information. Here we give aquick introduction to four useful features:history, tab completion,magic functions, and aliases.
Command history Like a UNIX shell, the IPython console supportscommand history. Type up and down tonavigate previously typedcommands:
In [1]: x = 10
In [2]:
In [2]: x = 10
Tab completion Tab completion, is a convenient way to explorethe structure of any object youre dealingwith. Simply typeobject_name. to view the objects attributes. Besides Python objectsand keywords,tab completion also works on file and directorynames.*
In [1]: x = 10
In [2]: x.x.bit_length x.denominator x.imag x.realx.conjugatex.from_bytes x.numerator x.to_bytes
Magic functions The console and the notebooks support so-calledmagic functions by prefixing a commandwith the % character. Forexample, the run and whos functions from the previous section aremagic functions.Note that, the setting automagic, which is enabledby default, allows you to omit the preceding % sign. Thus,you canjust type the magic function and it will work.
Other useful magic functions are:
%cd to change the current directory.
In [1]: cd /tmp/tmp
%cpaste allows you to paste code, especially code from websiteswhich has been prefixed with the stan-dard Python prompt (e.g.>>>) or with an ipython prompt, (e.g. in [3]):
In [2]: %cpastePasting code; enter '--' alone on the line tostop or use Ctrl-D.:>>> for i in range(3):
1.4. The workflow: interactive environments and text editors10
Scipy lecture notes, Edition 2017.1
:... print(i):--012
%timeit allows you to time the execution of short snippets usingthe timeit module from the standardlibrary:
In [3]: %timeit x = 1010000000 loops, best of 3: 39 ns perloop
See also:
Chapter on optimizing code
%debug allows you to enter post-mortem debugging. That is tosay, if the code you try to execute, raisesan exception, using%debug will enter the debugger at the point where the exception wasthrown.
In [4]: x === 10File "", line 1
x === 10^
SyntaxError: invalid syntax
In [5]: %debug> /.../IPython/core/compilerop.py(87)ast_parse()
86 and are passed to the built-in compile function."""---> 87return compile(source, filename, symbol, self.flags |PyCF_ONLY_AST, 1)
88
ipdb>locals(){'source': u'x === 10\n', 'symbol': 'exec','self':,'filename': ''}
See also:
Chapter on debugging
Aliases Furthermore IPython ships with various aliases whichemulate common UNIX command line toolssuch as ls to list files, cpto copy files and rm to remove files (a full list of aliases isshown when typing alias).
Getting help
The built-in cheat-sheet is accessible via the %quickref magicfunction.
A list of all available magic functions is shown when typing%magic.
1.4. The workflow: interactive environments and text editors11
CHAPTER2The Python language
Authors: Chris Burns, Christophe Combelles, EmmanuelleGouillart, Gal Varoquaux
Python for scientific computing
We introduce here the Python language. Only the bare minimumnecessary for getting started with Numpyand Scipy is addressedhere. To learn more about the language, consider going through theexcellent tutorialhttps://docs.python.org/tutorial. Dedicated booksare also available, such as http://www.diveintopython.net/.
Tip: Python is a programming language, as are C, Fortran, BASIC,PHP, etc. Some specific features of Pythonare as follows:
an interpreted (as opposed to compiled) language. Contrary toe.g. C or Fortran, one does not compilePython code before executingit. In addition, Python can be used interactively: many Pythoninter-preters are available, from which commands and scripts can beexecuted.
a free software released under an open-source license: Pythoncan be used and distributed free ofcharge, even for buildingcommercial software.
multi-platform: Python is available for all major operatingsystems, Windows, Linux/Unix, MacOS X,most likely your mobile phoneOS, etc.
a very readable language with clear non-verbose syntax
12
https://docs.python.org/tutorialhttp://www.diveintopython.net/http://www.diveintopython.net/
See Also
Control Tutorials for MATLAB and Simulink Code get stuck with nested loop and computer crashes after DC Motor Model - MATLAB & Simulink Control Tutorials for MATLAB and Simulink
Scipy lecture notes, Edition 2017.1
a language for which a large variety of high-quality packagesare available for various applications, fromweb frameworks toscientific computing.
a language very easy to interface with other languages, inparticular C and C++.
Some other features of the language are illustrated just below.For example, Python is an object-orientedlanguage, with dynamictyping (the same variable can contain objects of different typesduring thecourse of a program).
See https://www.python.org/about/ for more information aboutdistinguishing features of Python.
2.1 First steps
Start the Ipython shell (an enhanced interactive Pythonshell):
by typing ipython from a Linux/Mac terminal, or from the Windowscmd shell,
or by starting the program from a menu, e.g. in the Python(x,y)or EPD menu if you have installed oneof these scientific-Pythonsuites.
Tip: If you dont have Ipython installed on your computer, otherPython shells are available, such as the plainPython shell startedby typing python in a terminal, or the Idle interpreter. However,we advise to use theIpython shell because of its enhanced features,especially for interactive scientific computing.
Once you have started the interpreter, type
>>> print("Hello, world!")Hello, world!
Tip: The message Hello, world! is then displayed. You justexecuted your first Python instruction, congratu-lations!
To get yourself started, type the following stack ofinstructions
>>> a = 3>>> b = 2*a>>> type(b)
>>> print(b)6>>> a*b18>>> b ='hello'>>> type(b)
>>> b + b'hellohello'>>> 2*b'hellohello'
Tip: Two variables a and b have been defined above. Note thatone does not declare the type of a variablebefore assigning itsvalue. In C, conversely, one should write:
int a = 3;
2.1. First steps 13
https://www.python.org/about/http://www.pythonxy.com/http://store.enthought.com/
Scipy lecture notes, Edition 2017.1
In addition, the type of a variable may change, in the sensethat at one point in time it can be equal to a valueof a certaintype, and a second point in time, it can be equal to a value of adifferent type. b was first equal to aninteger, but it became equalto a string when it was assigned the value hello. Operations onintegers (b=2*a)are coded natively in Python, and so are someoperations on strings such as additions and multiplications,whichamount respectively to concatenation and repetition.
2.2 Basic types
2.2.1 Numerical types
Tip: Python supports the following numerical, scalar types:
Integer
>>> 1 + 12>>> a = 4>>> type(a)
Floats
>>> c = 2.1>>> type(c)
Complex
>>> a = 1.5 + 0.5j>>> a.real1.5>>>a.imag0.5>>> type(1. + 0j)
Booleans
>>> 3 > 4False>>> test = (3 >4)>>> testFalse>>> type(test)
Tip: A Python shell can therefore replace your pocketcalculator, with the basic arithmetic operations +, -, *,/, %(modulo) natively implemented
>>> 7 * 3.21.0>>> 2**101024>>> 8 %32
2.2. Basic types 14
Scipy lecture notes, Edition 2017.1
Type conversion (casting):
>>> float(1)1.0
Warning: Integer division
In Python 2:
>>> 3 / 21
In Python 3:
>>> 3 / 21.5
To be safe: use floats:
>>> 3 / 2.1.5
>>> a = 3>>> b = 2>>> a / b # InPython 21>>> a / float(b)1.5
Future behavior: to always get the behavior of Python3
>>> from __future__ import division>>> 3 /21.5
Tip: If you explicitly want integer division use //:
>>> 3.0 // 21.0
Note: The behaviour of the division operator has changed inPython 3.
2.2.2 Containers
Tip: Python provides many efficient types of containers, inwhich collections of objects can be stored.
Lists
Tip: A list is an ordered collection of objects, that may havedifferent types. For example:
>>> colors = ['red', 'blue', 'green', 'black','white']>>> type(colors)
Indexing: accessing individual objects contained in thelist:
2.2. Basic types 15
http://python3porting.com/preparing.html#use-instead-of-when-dividing-integers
Scipy lecture notes, Edition 2017.1
>>> colors[2]'green'
Counting from the end with negative indices:
>>> colors[-1]'white'>>> colors[-2]'black'
Warning: Indexing starts at 0 (as in C), not at 1 (as in Fortranor Matlab)!
Slicing: obtaining sublists of regularly-spaced elements:
>>> colors['red', 'blue', 'green', 'black','white']>>> colors[2:4]['green', 'black']
Warning: Note that colors[start:stop] contains the elements withindices i such as start>> colors['red', 'blue', 'green','black', 'white']>>> colors[3:]['black','white']>>> colors[:3]['red', 'blue', 'green']>>>colors[::2]['red', 'green', 'white']
Lists are mutable objects and can be modified:
>>> colors[0] = 'yellow'>>> colors['yellow','blue', 'green', 'black', 'white']>>> colors[2:4] =['gray', 'purple']>>> colors['yellow', 'blue', 'gray','purple', 'white']
Note: The elements of a list may have different types:
>>> colors = [3, -200, 'hello']>>> colors[3,-200, 'hello']>>> colors[1], colors[2](-200, 'hello')
Tip: For collections of numerical data that all have the sametype, it is often more efficient to use the arraytype provided bythe numpy module. A NumPy array is a chunk of memory containingfixed-sized items. With
2.2. Basic types 16
Scipy lecture notes, Edition 2017.1
NumPy arrays, operations on elements can be faster becauseelements are regularly spaced in memory andmore operations areperformed through specialized C functions instead of Pythonloops.
Tip: Python offers a large panel of functions to modify lists,or query them. Here are a few examples; for moredetails, seehttps://docs.python.org/tutorial/datastructures.html#more-on-lists
Add and remove elements:
>>> colors = ['red', 'blue', 'green', 'black','white']>>> colors.append('pink')>>>colors['red', 'blue', 'green', 'black', 'white','pink']>>> colors.pop() # removes and returns the lastitem'pink'>>> colors['red', 'blue', 'green', 'black','white']>>> colors.extend(['pink', 'purple']) # extendcolors, in-place>>> colors['red', 'blue', 'green','black', 'white', 'pink', 'purple']>>> colors =colors[:-2]>>> colors['red', 'blue', 'green', 'black','white']
Reverse:
>>> rcolors = colors[::-1]>>> rcolors['white','black', 'green', 'blue', 'red']>>> rcolors2 =list(colors)>>> rcolors2['red', 'blue', 'green', 'black','white']>>> rcolors2.reverse() # in-place>>>rcolors2['white', 'black', 'green', 'blue', 'red']
Concatenate and repeat lists:
>>> rcolors + colors['white', 'black', 'green', 'blue','red', 'red', 'blue', 'green', 'black', 'white']>>>rcolors * 2['white', 'black', 'green', 'blue', 'red', 'white','black', 'green', 'blue', 'red']
Tip: Sort:
>>> sorted(rcolors) # new object['black', 'blue','green', 'red', 'white']>>> rcolors['white', 'black','green', 'blue', 'red']>>> rcolors.sort() #in-place>>> rcolors['black', 'blue', 'green', 'red','white']
Methods and Object-Oriented Programming
The notation rcolors.method() (e.g. rcolors.append(3) andcolors.pop()) is our first example of
2.2. Basic types 17
https://docs.python.org/tutorial/datastructures.html#more-on-lists
Scipy lecture notes, Edition 2017.1
object-oriented programming (OOP). Being a list, the objectrcolors owns the method function that iscalled using the notation.. No further knowledge of OOP than understanding the notation . isnecessary forgoing through this tutorial.
Discovering methods:
Reminder: in Ipython: tab-completion (press tab)
In [28]: rcolors.rcolors.append rcolors.indexrcolors.removercolors.count rcolors.insertrcolors.reversercolors.extend rcolors.pop rcolors.sort
Strings
Different string syntaxes (simple, double or triple quotes):
s = 'Hello, how are you?'s = "Hi, what's up"s = '''Hello, #tripling the quotes allows the
how are you''' # string to span more than one lines ="""Hi,what's up?"""
In [1]: 'Hi, what'sup?'------------------------------------------------------------
File "", line 1'Hi, what's up?'
^SyntaxError: invalid syntax
The newline character is \n, and the tab character is \t.
Tip: Strings are collections like lists. Hence they can beindexed and sliced, using the same syntax and rules.
Indexing:
>>> a = "hello">>> a[0]'h'>>>a[1]'e'>>> a[-1]'o'
Tip: (Remember that negative indices correspond to counting fromthe right end.)
Slicing:
>>> a = "hello, world!">>> a[3:6] # 3rd to 6th(excluded) elements: elements 3, 4, 5'lo,'>>> a[2:10:2] #Syntax: a[start:stop:step]'lo o'
2.2. Basic types 18
Scipy lecture notes, Edition 2017.1
>>> a[::3] # every three characters, from beginning toend'hl r!'
Tip: Accents and special characters can also be handled inUnicode strings (seehttps://docs.python.org/tutorial/introduction.html#unicode-strings).
A string is an immutable object and it is not possible to modifyits contents. One may however create newstrings from the originalone.
In [53]: a = "hello, world!"In [54]: a[2] ='z'---------------------------------------------------------------------------Traceback(most recent call last):
File "", line 1, in TypeError: 'str' object does not supportitem assignment
In [55]: a.replace('l', 'z', 1)Out[55]: 'hezlo, world!'In [56]:a.replace('l', 'z')Out[56]: 'hezzo, worzd!'
Tip: Strings have many useful methods, such as a.replace as seenabove. Remember the a. object-orientednotation and use tabcompletion or help(str) to search for new methods.
See also:
Python offers advanced possibilities for manipulating strings,looking for patterns or formatting. The inter-ested reader isreferred tohttps://docs.python.org/library/stdtypes.html#string-methods andhttps://docs.python.org/library/string.html#new-string-formatting
String formatting:
>>> 'An integer: %i ; a float: %f ; another string: %s' % (1, 0.1, 'string')'An integer: 1; a float: 0.100000; anotherstring: string'
>>> i = 102>>> filename ='processing_of_dataset_%d .txt' % i>>>filename'processing_of_dataset_102.txt'
Dictionaries
Tip: A dictionary is basically an efficient table that maps keysto values. It is an unordered container
>>> tel = {'emmanuelle': 5752, 'sebastian':5578}>>> tel['francis'] = 5915>>>tel{'sebastian': 5578, 'francis': 5915, 'emmanuelle':5752}>>> tel['sebastian']5578>>>tel.keys()['sebastian', 'francis', 'emmanuelle']>>>tel.values()[5578, 5915, 5752]>>> 'francis' in telTrue
2.2. Basic types 19
https://docs.python.org/tutorial/introduction.html#unicode-stringshttps://docs.python.org/tutorial/introduction.html#unicode-stringshttps://docs.python.org/library/stdtypes.html#string-methodshttps://docs.python.org/library/string.html#new-string-formattinghttps://docs.python.org/library/string.html#new-string-formatting
Scipy lecture notes, Edition 2017.1
Tip: It can be used to conveniently store and retrieve valuesassociated with a name (a string for a date, aname, etc.). Seehttps://docs.python.org/tutorial/datastructures.html#dictionariesfor more information.
A dictionary can have keys (resp. values) with differenttypes:
>>> d = {'a':1, 'b':2, 3:'hello'}>>> d{'a': 1,3: 'hello', 'b': 2}
More container types
Tuples
Tuples are basically immutable lists. The elements of a tupleare written between parentheses, or just separatedby commas:
>>> t = 12345, 54321, 'hello!'>>>t[0]12345>>> t(12345, 54321, 'hello!')>>> u = (0,2)
Sets: unordered, unique items:
>>> s = set(('a', 'b', 'c', 'a'))>>>sset(['a', 'c', 'b'])>>> s.difference(('a','b'))set(['c'])
2.2.3 Assignment operator
Tip: Python library reference says:
Assignment statements are used to (re)bind names to values andto modify attributes or items ofmutable objects.
In short, it works as follows (simple assignment):
1. an expression on the right hand side is evaluated, thecorresponding object is created/obtained
2. a name on the left hand side is assigned, or bound, to ther.h.s. object
Things to note:
a single object can have several names bound to it:
In [1]: a = [1, 2, 3]In [2]: b = aIn [3]: aOut[3]: [1, 2, 3]In[4]: bOut[4]: [1, 2, 3]In [5]: a is bOut[5]: True
2.2. Basic types 20
https://docs.python.org/tutorial/datastructures.html#dictionarieshttps://docs.python.org/reference/simple_stmts.html#assignment-statements
Scipy lecture notes, Edition 2017.1
In [6]: b[1] = 'hi!'In [7]: aOut[7]: [1, 'hi!', 3]
to change a list in place, use indexing/slices:
In [1]: a = [1, 2, 3]In [3]: aOut[3]: [1, 2, 3]In [4]: a = ['a','b', 'c'] # Creates another object.In [5]: aOut[5]: ['a', 'b','c']In [6]: id(a)Out[6]: 138641676In [7]: a[:] = [1, 2, 3] #Modifies object in place.In [8]: aOut[8]: [1, 2, 3]In [9]:id(a)Out[9]: 138641676 # Same as in Out[6], yours willdiffer...
the key concept here is mutable vs. immutable
mutable objects can be changed in place
immutable objects cannot be modified once created
See also:
A very good and detailed explanation of the above issues can befound in David M. Beazleys article Types andObjects in Python.
2.3 Control Flow
Controls the order in which the code is executed.
2.3.1 if/elif/else
>>> if 2**2 == 4:... print('Obvious!')...Obvious!
Blocks are delimited by indentation
Tip: Type the following lines in your Python interpreter, and becareful to respect the indentation depth. TheIpython shellautomatically increases the indentation depth after a colon : sign;to decrease the indentationdepth, go four spaces to the left withthe Backspace key. Press the Enter key twice to leave the logicalblock.
>>> a = 10
>>> if a == 1:... print(1)... elif a == 2:...print(2)... else:... print('A lot')A lot
2.3. Control Flow 21
http://www.informit.com/articles/article.aspx?p=453682http://www.informit.com/articles/article.aspx?p=453682
Scipy lecture notes, Edition 2017.1
Indentation is compulsory in scripts as well. As an exercise,re-type the previous lines with the same indenta-tion in a scriptcondition.py, and execute the script with run condition.py inIpython.
2.3.2 for/range
Iterating with an index:
>>> for i in range(4):... print(i)0123
But most often, it is more readable to iterate over values:
>>> for word in ('cool', 'powerful', 'readable'):...print('Python is %s ' % word)Python is coolPython is powerfulPythonis readable
2.3.3 while/break/continue
Typical C-style while loop (Mandelbrot problem):
>>> z = 1 + 1j>>> while abs(z) < 100:... z= z**2 + 1>>> z(-134+352j)
More advanced features
break out of enclosing for/while loop:
>>> z = 1 + 1j
>>> while abs(z) < 100:... if z.imag == 0:...break... z = z**2 + 1
continue the next iteration of a loop.:
>>> a = [1, 0, 2, 4]>>> for element in a:...if element == 0:... continue... print(1. / element)1.00.50.25
2.3.4 Conditional Expressions
if
Evaluates to False:
2.3. Control Flow 22
Scipy lecture notes, Edition 2017.1
any number equal to zero (0, 0.0, 0+0j)
an empty container (list, tuple, set, dictionary, . . . )
False, None
Evaluates to True:
everything else
a == b Tests equality, with logics:
>>> 1 == 1.True
a is b Tests identity: both sides are the same object:
>>> 1 is 1.False
>>> a = 1>>> b = 1>>> a is bTrue
a in b For any collection b: b contains a
>>> b = [1, 2, 3]>>> 2 in bTrue>>> 5in bFalse
If b is a dictionary, this tests that a is a key of b.
2.3.5 Advanced iteration
Iterate over any sequence
You can iterate over any sequence (string, list, keys in adictionary, lines in a file, . . . ):
>>> vowels = 'aeiouy'
>>> for i in 'powerful':... if i in vowels:...print(i)oeu
>>> message = "Hello how are you?">>>message.split() # returns a list['Hello', 'how', 'are','you?']>>> for word in message.split():...print(word)...Hellohowareyou?
2.3. Control Flow 23
Scipy lecture notes, Edition 2017.1
Tip: Few languages (in particular, languages for scientificcomputing) allow to loop over anything but in-tegers/indices. WithPython it is possible to loop exactly over the objects of interestwithout bothering withindices you often dont care about. Thisfeature can often be used to make code more readable.
Warning: Not safe to modify the sequence you are iteratingover.
Keeping track of enumeration number
Common task is to iterate over a sequence while keeping track ofthe item number.
Could use while loop with a counter as above. Or a for loop:
>>> words = ('cool', 'powerful','readable')>>> for i in range(0, len(words)):... print((i,words[i]))(0, 'cool')(1, 'powerful')(2, 'readable')
But, Python provides a built-in function - enumerate - forthis:
>>> for index, item in enumerate(words):...print((index, item))(0, 'cool')(1, 'powerful')(2, 'readable')
Looping over a dictionary
Use items:
>>> d = {'a': 1, 'b':1.2, 'c':1j}
>>> for key, val in sorted(d.items()):... print('Key:%s has value: %s ' % (key, val))Key: a has value: 1Key: b hasvalue: 1.2Key: c has value: 1j
Note: The ordering of a dictionary in random, thus we usesorted() which will sort on the keys.
2.3.6 List Comprehensions
>>> [i**2 for i in range(4)][0, 1, 4, 9]
2.3. Control Flow 24
https://docs.python.org/2.7/library/functions.html#sorted
Scipy lecture notes, Edition 2017.1
Exercise
Compute the decimals of Pi using the Wallis formula:
= 2
i=1
4i 2
4i 2 1
2.4 Defining functions
2.4.1 Function definition
In [56]: def test():....: print('in testfunction')....:....:
In [57]: test()in test function
Warning: Function blocks must be indented as other control-flowblocks.
2.4.2 Return statement
Functions can optionally return values.
In [6]: def disk_area(radius):...: return 3.14 * radius *radius...:
In [8]: disk_area(1.5)Out[8]: 7.0649999999999995
See Also
Linux Basic Commands: grep ls cd cat cp rm scp
Note: By default, functions return None.
Note: Note the syntax to define a function:
the def keyword;
is followed by the functions name, then
the arguments of the function are given between parenthesesfollowed by a colon.
the function body;
and return object for optionally returning values.
2.4.3 Parameters
Mandatory parameters (positional arguments)
2.4. Defining functions 25
Scipy lecture notes, Edition 2017.1
In [81]: def double_it(x):....: return x * 2....:
In [82]: double_it(3)Out[82]: 6
In [83]:double_it()---------------------------------------------------------------------------Traceback(most recent call last):
File "", line 1, in TypeError: double_it() takes exactly 1argument (0 given)
Optional parameters (keyword or named arguments)
In [84]: def double_it(x=2):....: return x * 2....:
In [85]: double_it()Out[85]: 4
In [86]: double_it(3)Out[86]: 6
Keyword arguments allow you to specify default values.
Warning: Default values are evaluated when the function isdefined, not when it is called. This can beproblematic when usingmutable types (e.g. dictionary or list) and modifying them in thefunction body,since the modifications will be persistent acrossinvocations of the function.
Using an immutable type in a keyword argument:
In [124]: bigx = 10
In [125]: def double_it(x=bigx):.....: return x * 2.....:
In [126]: bigx = 1e9 # Now really big
In [128]: double_it()Out[128]: 20
Using an mutable type in a keyword argument (and modifying itinside the function body):
In [2]: def add_to_dict(args={'a': 1, 'b': 2}):...: for i inargs.keys():...: args[i] += 1...: print args...:
In [3]: add_to_dictOut[3]:
In [4]: add_to_dict(){'a': 2, 'b': 3}
In [5]: add_to_dict(){'a': 3, 'b': 4}
In [6]: add_to_dict(){'a': 4, 'b': 5}
2.4. Defining functions 26
Scipy lecture notes, Edition 2017.1
Tip: More involved example implementing pythons slicing:
In [98]: def slicer(seq, start=None, stop=None, step=None):....:"""Implement basic python slicing."""....: returnseq[start:stop:step]....:
In [101]: rhyme = 'one fish, two fish, red fish, bluefish'.split()
In [102]: rhymeOut[102]: ['one', 'fish,', 'two', 'fish,', 'red','fish,', 'blue', 'fish']
In [103]: slicer(rhyme)Out[103]: ['one', 'fish,', 'two','fish,', 'red', 'fish,', 'blue', 'fish']
In [104]: slicer(rhyme, step=2)Out[104]: ['one', 'two', 'red','blue']
In [105]: slicer(rhyme, 1, step=2)Out[105]: ['fish,', 'fish,','fish,', 'fish']
In [106]: slicer(rhyme, start=1, stop=4, step=2)Out[106]:['fish,', 'fish,']
The order of the keyword arguments does not matter:
In [107]: slicer(rhyme, step=2, start=1, stop=4)Out[107]:['fish,', 'fish,']
but it is good practice to use the same ordering as thefunctions definition.
Keyword arguments are a very convenient feature for definingfunctions with a variable number of arguments,especially whendefault values are to be used in most calls to the function.
2.4.4 Passing by value
Tip: Can you modify the value of a variable inside a function?Most languages (C, Java, . . . ) distinguishpassing by value andpassing by reference. In Python, such a distinction is somewhatartificial, and it is abit subtle whether your variables are goingto be modified or not. Fortunately, there exist clear rules.
Parameters to functions are references to objects, which arepassed by value. When you pass a variable to afunction, pythonpasses the reference to the object to which the variable refers(the value). Not the variableitself.
If the value passed in a function is immutable, the functiondoes not modify the callers variable. If the valueis mutable, thefunction may modify the callers variable in-place:
>>> def try_to_modify(x, y, z):... x = 23...y.append(42)... z = [99] # new reference... print(x)... print(y)...print(z)...
2.4. Defining functions 27
Scipy lecture notes, Edition 2017.1
>>> a = 77 # immutable variable>>> b = [99] #mutable variable>>> c = [28]>>> try_to_modify(a,b, c)23[99, 42][99]>>> print(a)77>>> print(b)[99,42]>>> print(c)[28]
Functions have a local variable table called a localnamespace.
The variable x only exists within the functiontry_to_modify.
2.4.5 Global variables
Variables declared outside the function can be referenced withinthe function:
In [114]: x = 5
In [115]: def addx(y):.....: return x + y.....:
In [116]: addx(10)Out[116]: 15
But these global variables cannot be modified within thefunction, unless declared global in the function.
This doesnt work:
In [117]: def setx(y):.....: x = y.....: print('x is %d ' %x).....:.....:
In [118]: setx(10)x is 10
In [120]: xOut[120]: 5
This works:
In [121]: def setx(y):.....: global x.....: x = y.....: print('xis %d ' % x).....:.....:
In [122]: setx(10)x is 10
In [123]: xOut[123]: 10
2.4. Defining functions 28
Scipy lecture notes, Edition 2017.1
2.4.6 Variable number of parameters
Special forms of parameters:
*args: any number of positional arguments packed into atuple
**kwargs: any number of keyword arguments packed into adictionary
In [35]: def variable_args(*args, **kwargs):....: print 'argsis', args....: print 'kwargs is', kwargs....:
In [36]: variable_args('one', 'two', x=1, y=2, z=3)args is('one', 'two')kwargs is {'y': 2, 'x': 1, 'z': 3}
2.4.7 Docstrings
Documentation about what the function does and its parameters.General convention:
In [67]: def funcname(params):....: """Concise one-line sentencedescribing the function.....:....: Extended summary which cancontain multiple paragraphs.....: """....: # function body....:pass....:
In [68]: funcname?Type: functionBase Class: type'function'>String Form: Namespace: InteractiveFile: Definition:funcname(params)Docstring:
Concise one-line sentence describing the function.
Extended summary which can contain multiple paragraphs.
Note: Docstring guidelines
For the sake of standardization, the Docstring Conventionswebpage documents the semantics and conven-tions associated withPython docstrings.
Also, the Numpy and Scipy modules have defined a precisestandard for documenting scientific func-tions, that you may wantto follow for your own functions, with a Parameters section, anExamples sec-tion, etc. Seehttp://projects.scipy.org/numpy/wiki/CodingStyleGuidelines#docstring-standardandhttp://projects.scipy.org/numpy/browser/trunk/doc/example.py#L37
2.4.8 Functions are objects
Functions are first-class objects, which means they can be:
assigned to a variable
an item in a list (or any collection)
2.4. Defining functions 29
https://www.python.org/dev/peps/pep-0257http://projects.scipy.org/numpy/wiki/CodingStyleGuidelines#docstring-standardhttp://projects.scipy.org/numpy/browser/trunk/doc/example.py#L37http://projects.scipy.org/numpy/browser/trunk/doc/example.py#L37
Scipy lecture notes, Edition 2017.1
passed as an argument to another function.
In [38]: va = variable_args
In [39]: va('three', x=1, y=2)args is ('three',)kwargs is {'y':2, 'x': 1}
2.4.9 Methods
Methods are functions attached to objects. Youve seen these inour examples on lists, dictionaries, strings,etc. . .
2.4.10 Exercises
Exercise: Fibonacci sequence
Write a function that displays the n first terms of theFibonacci sequence, defined by:U0 = 0U1 = 1Un+2 =Un+1 +Un
Exercise: Quicksort
Implement the quicksort algorithm, as defined by wikipedia
function quicksort(array)var list less, greaterif length(array)< 2
return arrayselect and remove a pivot value pivot from arrayforeach x in array
if x < pivot + 1 then append x to lesselse append x togreater
return concatenate(quicksort(less), pivot,quicksort(greater))
2.5 Reusing code: scripts and modules
For now, we have typed all instructions in the interpreter. Forlonger sets of instructions we need to changetrack and write thecode in text files (using a text editor), that we will call eitherscripts or modules. Use your fa-vorite text editor (provided itoffers syntax highlighting for Python), or the editor that comeswith the ScientificPython Suite you may be using.
2.5.1 Scripts
Tip: Let us first write a script, that is a file with a sequenceof instructions that are executed each time the scriptis called.Instructions may be e.g. copied-and-pasted from the interpreter(but take care to respect indentationrules!).
2.5. Reusing code: scripts and modules 30
Scipy lecture notes, Edition 2017.1
The extension for Python files is .py. Write or copy-and-pastethe following lines in a file called test.py
message = "Hello how are you?"for word in message.split():
print word
Tip: Let us now execute the script interactively, that is insidethe Ipython interpreter. This is maybe the mostcommon use ofscripts in scientific computing.
Note: in Ipython, the syntax to execute a script is %runscript.py. For example,
In [1]: %run test.pyHellohowareyou?
In [2]: messageOut[2]: 'Hello how are you?'
The script has been executed. Moreover the variables defined inthe script (such as message) are now availableinside theinterpreters namespace.
Tip: Other interpreters also offer the possibility to executescripts (e.g., execfile in the plain Python inter-preter,etc.).
It is also possible In order to execute this script as astandalone program, by executing the script inside a shellterminal(Linux/Mac console or cmd Windows console). For example, if we arein the same directory as thetest.py file, we can execute this in aconsole:
$ python test.pyHellohowareyou?
Tip: Standalone scripts may also take command-line arguments
In file.py:
import sysprint sys.argv
$ python file.py test arguments['file.py', 'test','arguments']
Warning: Dont implement option parsing yourself. Use modulessuch as optparse, argparse or:moddocopt.
2.5. Reusing code: scripts and modules 31
https://docs.python.org/2.7/library/optparse.html#module-optparsehttps://docs.python.org/2.7/library/argparse.html#module-argparse
Scipy lecture notes, Edition 2017.1
2.5.2 Importing objects from modules
In [1]: import os
In [2]: osOut[2]:
In [3]:os.listdir('.')Out[3]:['conf.py','basic_types.rst','control_flow.rst','functions.rst','python_language.rst','reusing.rst','file_io.rst','exceptions.rst','workflow.rst','index.rst']
And also:
In [4]: from os import listdir
Importing shorthands:
In [5]: import numpy as np
Warning:
from os import *
This is called the star import and please, Do not use it
Makes the code harder to read and understand: where do symbolscome from?
Makes it impossible to guess the functionality by the contextand the name (hint: os.name is thename of the OS), and to profitusefully from tab completion.
Restricts the variable names you can use: os.name might overridename, or vise-versa.
Creates possible name clashes between modules.
Makes the code impossible to statically check for undefinedsymbols.
Tip: Modules are thus a good way to organize code in ahierarchical way. Actually, all the scientific computingtools weare going to use are modules:
>>> import numpy as np # data arrays>>>np.linspace(0, 10, 6)array([ 0., 2., 4., 6., 8., 10.])>>>import scipy # scientific computing
2.5.3 Creating modules
Tip: If we want to write larger and better organized programs(compared to simple scripts), where someobjects are defined,(variables, functions, classes) and that we want to reuse severaltimes, we have to create
2.5. Reusing code: scripts and modules 32
Scipy lecture notes, Edition 2017.1
our own modules.
Let us create a module demo contained in the file demo.py:
"A demo module."
def print_b():"Prints b."print 'b'
def print_a():"Prints a."print 'a'
c = 2d = 2
Tip: In this file, we defined two functions print_a and print_b.Suppose we want to call the print_afunction from the interpreter.We could execute the file as a script, but since we just want tohave access to thefunction print_a, we are rather going to importit as a module. The syntax is as follows.
In [1]: import demo
In [2]: demo.print_a()a
In [3]: demo.print_b()b
Importing the module gives access to its objects, using themodule.object syntax. Dont forget to put themodules name before theobjects name, otherwise Python wont recognize the instruction.
Introspection
In [4]: demo?Type: moduleBase Class: String Form: Namespace:InteractiveFile:/home/varoquau/Projects/Python_talks/scipy_2009_tutorial/source/demo.pyDocstring:
A demo module.
In [5]: whodemo
In [6]: whosVariable TypeData/Info------------------------------demo module
In [7]:dir(demo)Out[7]:['__builtins__','__doc__','__file__','__name__',
2.5. Reusing code: scripts and modules 33
Scipy lecture notes, Edition 2017.1
'__package__','c','d','print_a','print_b']
In [8]: demo.demo.c demo.print_a demo.pydemo.d demo.print_bdemo.pyc
Importing objects from modules into the main namespace
In [9]: from demo import print_a, print_b
In [10]: whosVariable TypeData/Info--------------------------------demo module print_afunction print_b function
In [11]: print_a()a
Warning: Module caching
Modules are cached: if you modify demo.py and re-import it inthe old session, you will get theold one.
Solution:
In [10]: reload(demo)
In Python3 instead reload is not builtin, so you have to importthe importlib module first and then do:
In [10]: importlib.reload(demo)
2.5.4 __main__ and module loading
Tip: Sometimes we want code to be executed when a module is rundirectly, but not when it is imported byanother module. if __name__== '__main__' allows us to check whether the module is being rundirectly.
File demo2.py:
def print_b():"Prints b."print 'b'
def print_a():"Prints a."print 'a'
# print_b() runs on importprint_b()
if __name__ == '__main__':# print_a() is only executed when themodule is run directly.print_a()
2.5. Reusing code: scripts and modules 34
Scipy lecture notes, Edition 2017.1
Importing it:
In [11]: import demo2b
In [12]: import demo2
Running it:
In [13]: %run demo2ba
2.5.5 Scripts or modules? How to organize your code
Note: Rule of thumb
Sets of instructions that are called several times should bewritten inside functions for better codereusability.
Functions (or other bits of code) that are called from severalscripts should be written inside a module,so that only the moduleis imported in the different scripts (do not copy-and-paste yourfunctions in thedifferent scripts!).
How modules are found and imported
When the import mymodule statement is executed, the modulemymodule is searched in a given list of direc-tories. This listincludes a list of installation-dependent default path (e.g.,/usr/lib/python) as well as thelist of directories specified by theenvironment variable PYTHONPATH.
The list of directories searched by Python is given by thesys.path variable
In [1]: import sys
In [2]:sys.pathOut[2]:['','/home/varoquau/.local/bin','/usr/lib/python2.7','/home/varoquau/.local/lib/python2.7/site-packages','/usr/lib/python2.7/dist-packages','/usr/local/lib/python2.7/dist-packages',...]
Modules must be located in the search path, therefore youcan:
write your own modules within directories already defined in thesearch path (e.g. $HOME/.local/lib/python2.7/dist-packages). Youmay use symbolic links (on Linux) to keep the code somewhereelse.
modify the environment variable PYTHONPATH to include thedirectories containing the user-definedmodules.
Tip: On Linux/Unix, add the following line to a file read by theshell at startup (e.g. /etc/profile, .profile)
exportPYTHONPATH=$PYTHONPATH:/home/emma/user_defined_modules
2.5. Reusing code: scripts and modules 35
Scipy lecture notes, Edition 2017.1
On Windows, http://support.microsoft.com/kb/310519 explains howto handle environment variables.
or modify the sys.path variable itself within a Pythonscript.
Tip:
import sysnew_path = '/home/emma/user_defined_modules'ifnew_path not in sys.path:
sys.path.append(new_path)
This method is not very robust, however, because it makes thecode less portable (user-dependent path)and because you have to addthe directory to your sys.path each time you want to import from amodulein this directory.
See also:
See https://docs.python.org/tutorial/modules.html for moreinformation about modules.
2.5.6 Packages
A directory that contains many modules is called a package. Apackage is a module with submodules (whichcan have submodulesthemselves, etc.). A special file called __init__.py (which may beempty) tells Pythonthat the directory is a Python package, fromwhich modules can be imported.
$ lscluster/ io/ README.txt@ stsci/__config__.py@ LATEST.txt@setup.py@ __svn_version__.py@__config__.pyc lib/ setup.pyc__svn_version__.pycconstants/ linalg/ setupscons.py@THANKS.txt@fftpack/ linsolve/ setupscons.pycTOCHANGE.txt@__init__.py@ maxentropy/ signal/version.py@__init__.pyc misc/ sparse/ version.pycINSTALL.txt@ndimage/ spatial/ weave/integrate/ odr/ special/interpolate/optimize/ stats/$ cd ndimage$ lsdoccer.py@ fourier.pycinterpolation.py@ morphology.pyc setup.pycdoccer.pyc info.py@interpolation.pyc [emailprotected]@ info.pycmeasurements.py@ [emailprotected]__init__.py@ measurements.pyc _ni_support.pyc tests/fourier.py@__init__.pyc morphology.py@ setup.py@
From Ipython:
In [1]: import scipy
In [2]: scipy.__file__Out[2]:'/usr/lib/python2.6/dist-packages/scipy/__init__.pyc'
In [3]: import scipy.version
In [4]: scipy.version.versionOut[4]: '0.7.0'
In [5]: import scipy.ndimage.morphology
2.5. Reusing code: scripts and modules 36
http://support.microsoft.com/kb/310519https://docs.python.org/tutorial/modules.html
Scipy lecture notes, Edition 2017.1
In [6]: from scipy.ndimage import morphology
In [17]: morphology.binary_dilation?Type: functionBase Class:String Form: Namespace: InteractiveFile:/usr/lib/python2.6/dist-packages/scipy/ndimage/morphology.pyDefinition:morphology.binary_dilation(input, structure=None,iterations=1,mask=None, output=None, border_value=0,origin=0,brute_force=False)Docstring:
Multi-dimensional binary dilation with the given structure.
An output array can optionally be provided. The originparametercontrols the placement of the filter. If no structuringelement isprovided an element is generated with a squaredconnectivity equalto one. The dilation operation is repeatediterations times. Ifiterations is less than 1, the dilation isrepeated until theresult does not change anymore. If a mask isgiven, only thoseelements with a true value at the correspondingmask element aremodified at each iteration.
2.5.7 Good practices
Use meaningful object names
Indentation: no choice!
Tip: Indenting is compulsory in Python! Every command blockfollowing a colon bears an additionalindentation level with respectto the previous line with a colon. One must therefore indent afterdeff(): or while:. At the end of such logical blocks, one decreasesthe indentation depth (and re-increasesit if a new block isentered, etc.)
Strict respect of indentation is the price to pay for gettingrid of { or ; characters that delineate logicalblocks in otherlanguages. Improper indentation leads to errors such as
------------------------------------------------------------IndentationError:unexpected indent (test.py, line 2)
All this indentation business can be a bit confusing in thebeginning. However, with the clear indenta-tion, and in the absenceof extra characters, the resulting code is very nice to readcompared to otherlanguages.
Indentation depth: Inside your text editor, you may choose toindent with any positive number of spaces(1, 2, 3, 4, . . . ).However, it is considered good practice to indent with 4 spaces.You may configure youreditor to map the Tab key to a 4-spaceindentation.
Style guidelines
Long lines: you should not write very long lines that span overmore than (e.g.) 80 characters. Long linescan be broken with the \character
>>> long_line = "Here is a very very long line \...that we break in two parts."
Spaces
Write well-spaced code: put whitespaces after commas, aroundarithmetic operators, etc.:
2.5. Reusing code: scripts and modules 37
Scipy lecture notes, Edition 2017.1
>>> a = 1 # yes>>> a=1 # too cramped
A certain number of rules for writing beautiful code (and moreimportantly using the same conven-tions as anybody else!) are givenin the Style Guide for Python Code.
Quick read
If you want to do a first quick pass through the Scipy lecturesto learn the ecosystem, you can directly skipto the next chapter:NumPy: creating and manipulating numerical data.
The remainder of this chapter is not necessary to follow therest of the intro part. But be sure to come backand finish thischapter later.
2.6 Input and Output
To be exhaustive, here are some information about input andoutput in Python. Since we will use the Numpymethods to read andwrite files, you may skip this chapter at first reading.
We write or read strings to/from files (other types must beconverted to strings). To write in a file:
>>> f = open('workfile', 'w') # opens the workfilefile>>> type(f)
>>> f.write('This is a test \nand anothertest')>>> f.close()
To read from a file
In [1]: f = open('workfile', 'r')
In [2]: s = f.read()
In [3]: print(s)This is a testand another test
In [4]: f.close()
See also:
For more details:https://docs.python.org/tutorial/inputoutput.html
2.6.1 Iterating over a file
In [6]: f = open('workfile', 'r')
In [7]: for line in f:...: print line...:
This is a test
and another test
In [8]: f.close()
2.6. Input and Output 38
https://www.python.org/dev/peps/pep-0008https://docs.python.org/tutorial/inputoutput.html
Scipy lecture notes, Edition 2017.1
File modes
Read-only: r
Write-only: w
Note: Create a new file or overwrite existing file.
Append a file: a
Read and Write: r+
Binary mode: b
Note: Use for binary files, especially on Windows.
2.7 Standard Library
Note: Reference document for this section:
The Python Standard Library documentation:https://docs.python.org/library/index.html
Python Essential Reference, David Beazley, Addison-WesleyProfessional
2.7.1 os module: operating system functionality
A portable way of using operating system dependentfunctionality.
Directory and file manipulation
Current directory:
In [17]: os.getcwd()Out[17]:'/Users/cburns/src/scipy2009/scipy_2009_tutorial/source'
List a directory:
In [31]:os.listdir(os.curdir)Out[31]:['.index.rst.swo','.python_language.rst.swp','.view_array.py.swp','_static','_templates','basic_types.rst','conf.py','control_flow.rst','debugging.rst',...
Make a directory:
In [32]: os.mkdir('junkdir')
In [33]: 'junkdir' in os.listdir(os.curdir)Out[33]: True
Rename the directory:
2.7. Standard Library 39
https://docs.python.org/library/index.html
Scipy lecture notes, Edition 2017.1
In [36]: os.rename('junkdir', 'foodir')
In [37]: 'junkdir' in os.listdir(os.curdir)Out[37]: False
In [38]: 'foodir' in os.listdir(os.curdir)Out[38]: True
In [41]: os.rmdir('foodir')
In [42]: 'foodir' in os.listdir(os.curdir)Out[42]: False
Delete a file:
In [44]: fp = open('junk.txt', 'w')
In [45]: fp.close()
In [46]: 'junk.txt' in os.listdir(os.curdir)Out[46]: True
In [47]: os.remove('junk.txt')
In [48]: 'junk.txt' in os.listdir(os.curdir)Out[48]: False
os.path: path manipulations
os.path provides common operations on pathnames.
In [70]: fp = open('junk.txt', 'w')
In [71]: fp.close()
In [72]: a = os.path.abspath('junk.txt')
In [73]: aOut[73]:'/Users/cburns/src/scipy2009/scipy_2009_tutorial/source/junk.txt'
In [74]: os.path.split(a)Out[74]:('/Users/cburns/src/scipy2009/scipy_2009_tutorial/source',
'junk.txt')
In [78]: os.path.dirname(a)Out[78]:'/Users/cburns/src/scipy2009/scipy_2009_tutorial/source'
In [79]: os.path.basename(a)Out[79]: 'junk.txt'
In [80]: os.path.splitext(os.path.basename(a))Out[80]: ('junk','.txt')
In [84]: os.path.exists('junk.txt')Out[84]: True
In [86]: os.path.isfile('junk.txt')Out[86]: True
In [87]: os.path.isdir('junk.txt')Out[87]: False
2.7. Standard Library 40
Scipy lecture notes, Edition 2017.1
In [88]: os.path.expanduser('~/local')Out[88]:'/Users/cburns/local'
In [92]: os.path.join(os.path.expanduser('~'), 'local','bin')Out[92]: '/Users/cburns/local/bin'
Running an external command
In [8]: os.system('ls')basic_types.rst demo.py functions.rstpython_language.rst standard_library.rstcontrol_flow.rstexceptions.rst io.rst python-logo.pngdemo2.py first_steps.rstoop.rst reusing_code.rst
Note: Alternative to os.system
A noteworthy alternative to os.system is the sh module. Whichprovides much more convenient ways toobtain the output, errorstream and exit code of the external command.
In [20]: import shIn [20]: com = sh.ls()
In [21]: print combasic_types.rst exceptions.rst oop.rststandard_library.rstcontrol_flow.rst first_steps.rstpython_language.rstdemo2.py functions.rst python-logo.pngdemo.pyio.rst reusing_code.rst
In [22]: print com.exit_code0In [23]: type(com)Out[23]:sh.RunningCommand
Walking a directory
os.path.walk generates a list of filenames in a directorytree.
In [10]: for dirpath, dirnames, filenames inos.walk(os.curdir):....: for fp in filenames:....: printos.path.abspath(fp)....:....:
/Users/cburns/src/scipy2009/scipy_2009_tutorial/source/.index.rst.swo/Users/cburns/src/scipy2009/scipy_2009_tutorial/source/.view_array.py.swp/Users/cburns/src/scipy2009/scipy_2009_tutorial/source/basic_types.rst/Users/cburns/src/scipy2009/scipy_2009_tutorial/source/conf.py/Users/cburns/src/scipy2009/scipy_2009_tutorial/source/control_flow.rst...
Environment variables:
In [9]: import os
In [11]: os.environ.keys()Out[11]:
2.7. Standard Library 41
http://amoffat.github.com/sh/
Scipy lecture notes, Edition 2017.1
['_','FSLDIR','TERM_PROGRAM_VERSION','FSLREMOTECALL','USER','HOME','PATH','PS1','SHELL','EDITOR','WORKON_HOME','PYTHONPATH',...
In [12]: os.environ['PYTHONPATH']Out[12]:'.:/Users/cburns/src/utils:/Users/cburns/src/nitools:/Users/cburns/local/lib/python2.5/site-packages/:/usr/local/lib/python2.5/site-packages/:/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5'
In [16]: os.getenv('PYTHONPATH')Out[16]:'.:/Users/cburns/src/utils:/Users/cburns/src/nitools:/Users/cburns/local/lib/python2.5/site-packages/:/usr/local/lib/python2.5/site-packages/:/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5'
2.7.2 shutil: high-level file operations
The shutil provides useful file operations:
shutil.rmtree: Recursively delete a directory tree.
shutil.move: Recursively move a file or directory to anotherlocation.
shutil.copy: Copy files or directories.
2.7.3 glob: Pattern matching on files
The glob module provides convenient file pattern matching.
Find all files ending in .txt:
In [18]: import glob
In [19]: glob.glob('*.txt')Out[19]: ['holy_grail.txt','junk.txt', 'newfile.txt']
2.7.4 sys module: system-specific information
System-specific information related to the Pythoninterpreter.
Which version of python are you running and where is itinstalled:
In [117]: sys.platformOut[117]: 'darwin'
In [118]: sys.versionOut[118]: '2.5.2 (r252:60911, Feb 22 2008,07:57:53) \n
[GCC 4.0.1 (Apple Computer, Inc. build 5363)]'
2.7. Standard Library 42
Scipy lecture notes, Edition 2017.1
In [119]: sys.prefixOut[119]:'/Library/Frameworks/Python.framework/Versions/2.5'
List of command line arguments passed to a Python script:
In [100]: sys.argvOut[100]:['/Users/cburns/local/bin/ipython']
sys.path is a list of strings that specifies the search path formodules. Initialized from PYTHONPATH:
In [121]:sys.pathOut[121]:['','/Users/cburns/local/bin','/Users/cburns/local/lib/python2.5/site-packages/grin-1.1-py2.5.egg','/Users/cburns/local/lib/python2.5/site-packages/argparse-0.8.0-py2.5.egg','/Users/cburns/local/lib/python2.5/site-packages/urwid-0.9.7.1-py2.5.egg','/Users/cburns/local/lib/python2.5/site-packages/yolk-0.4.1-py2.5.egg','/Users/cburns/local/lib/python2.5/site-packages/virtualenv-1.2-py2.5.egg',...
2.7.5 pickle: easy persistence
Useful to store arbitrary objects to a file. Not safe orfast!
In [1]: import pickle
In [2]: l = [1, None, 'Stan']
In [3]: pickle.dump(l, file('test.pkl', 'w'))
In [4]: pickle.load(file('test.pkl'))Out[4]: [1, None,'Stan']
Exercise
Write a program to search your PYTHONPATH for the modulesite.py.
path_site
2.8 Exception handling in Python
It is likely that you have raised Exceptions if you have typedall the previous commands of the tutorial. Forexample, you may haveraised an exception if you entered a command with a typo.
Exceptions are raised by different kinds of errors arising whenexecuting Python code. In your own code, youmay also catch errors,or define custom error types. You may want to look at thedescriptions of the the built-inExceptions when looking for theright exception type.
2.8.1 Exceptions
Exceptions are raised by errors in Python:
2.8. Exception handling in Python 43
https://docs.python.org/2/library/exceptions.htmlhttps://docs.python.org/2/library/exceptions.html
Scipy lecture notes, Edition 2017.1
In [1]:1/0---------------------------------------------------------------------------ZeroDivisionError:integer division or modulo by zero
In [2]: 1 +'e'---------------------------------------------------------------------------TypeError:unsupported operand type(s) for +: 'int' and 'str'
In [3]: d = {1:1, 2:2}
In [4]:d[3]---------------------------------------------------------------------------KeyError:3
In [5]: l = [1, 2, 3]
In [6]:l[4]---------------------------------------------------------------------------IndexError:list index out of range
In [7]:l.foobar---------------------------------------------------------------------------AttributeError:'list' object has no attribute 'foobar'
As you can see, there are different types of exceptions fordifferent errors.
2.8.2 Catching exceptions
try/except
In [10]: while True:....: try:....: x = int(raw_input('Pleaseenter a number: '))....: break....: except ValueError:....:print('That was no valid number. Try again...')....:
Please enter a number: aThat was no valid number. Tryagain...Please enter a number: 1
In [9]: xOut[9]: 1
try/finally
In [10]: try:....: x = int(raw_input('Please enter a number:'))....: finally:....: print('Thank you for yourinput')....:....:
Please enter a number: aThank you for yourinput---------------------------------------------------------------------------ValueError:invalid literal for int() with base 10: 'a'
Important for resource management (e.g. closing a file)
2.8. Exception handling in Python 44
Scipy lecture notes, Edition 2017.1
Easier to ask for forgiveness than for permission
In [11]: def print_sorted(collection):....: try:....:collection.sort()....: except AttributeError:....: pass....:print(collection)....:....:
In [12]: print_sorted([1, 3, 2])[1, 2, 3]
In [13]: print_sorted(set((1, 3, 2)))set([1, 2, 3])
In [14]: print_sorted('132')132
2.8.3 Raising exceptions
Capturing and reraising an exception:
In [15]: def filter_name(name):....: try:....: name =name.encode('ascii')....: except UnicodeError as e:....: if name =='Gal':....: print('OK, Gal')....: else:....: raise e....: returnname....:
In [16]: filter_name('Gal')OK, GalOut[16]: 'Ga\xc3\xabl'
In [17]:filter_name('Stfan')---------------------------------------------------------------------------UnicodeDecodeError:'ascii' codec can't decode byte 0xc3 in position 2: ordinal notin
,range(128)
Exceptions to pass messages between parts of the code:
In [17]: def achilles_arrow(x):....: if abs(x - 1) <1e-3:....: raise StopIteration....: x = 1 - (1-x)/2.....: returnx....:
In [18]: x = 0
In [19]: while True:....: try:....: x = achilles_arrow(x)....:except StopIteration:....: break....:
2.8. Exception handling in Python 45
Scipy lecture notes, Edition 2017.1
....:
In [20]: xOut[20]: 0.9990234375
Use exceptions to notify certain conditions are met (e.g.StopIteration) or not (e.g. custom error raising)
2.9 Object-oriented programming (OOP)
Python supports object-oriented programming (OOP). The goals ofOOP are:
to organize the code, and
to re-use code in similar contexts.
Here is a small example: we create a Student class, which is anobject gathering several custom functions(methods) and variables(attributes), we will be able to use:
>>> class Student(object):... def __init__(self,name):... self.name = name... def set_age(self, age):... self.age =age... def set_major(self, major):... self.major =major...>>> anna = Student('anna')>>>anna.set_age(21)>>> anna.set_major('physics')
In the previous example, the Student class has __init__, set_ageand set_major methods. Its at-tributes are name, age and major. Wecan call these methods and attributes with the followingnotation:classinstance.method or classinstance.attribute. The__init__ constructor is a special method wecall with: MyClass(initparameters if any).
Now, suppose we want to create a new class MasterStudent withthe same methods and attributes as the pre-vious one, but with anadditional internship attribute. We wont copy the previous class,but inherit fromit:
>>> class MasterStudent(Student):... internship ='mandatory, from March to June'...>>> james =MasterStudent('james')>>> james.internship'mandatory, fromMarch to June'>>> james.set_age(23)>>>james.age23
The MasterStudent class inherited from the Student attributesand methods.
Thanks to classes and object-oriented programming, we canorganize code with different classes correspond-ing to differentobjects we encounter (an Experiment class, an Image class, a Flowclass, etc.), with their ownmethods and attributes. Then we can useinheritance to consider variations around a base class andre-usecode. Ex : from a Flow base class, we can create derivedStokesFlow, TurbulentFlow, PotentialFlow, etc.
2.9. Object-oriented programming (OOP) 46
CHAPTER3NumPy: creating and manipulating
numerical data
Authors: Emmanuelle Gouillart, Didrik Pinte, Gal Varoquaux, andPauli Virtanen
This chapter gives an overview of NumPy, the core tool forperformant numerical computing with Python.
3.1 The NumPy array object
Section contents
What are NumPy and NumPy arrays?
Creating arrays
Basic data types
Basic visualization
Indexing and slicing
Copies and views
Fancy indexing
3.1.1 What are NumPy and NumPy arrays?
47
Scipy lecture notes, Edition 2017.1
NumPy arrays
Python objects
high-level number objects: integers, floating point
containers: lists (costless insertion and append), dictionaries(fast lookup)
NumPy provides
extension package to Python for multi-dimensional arrays
closer to hardware (efficiency)
designed for scientific computation (convenience)
Also known as array oriented computing
>>> import numpy as np>>> a = np.array([0, 1,2, 3])>>> aarray([0, 1, 2, 3])
Tip: For example, An array containing:
values of an experiment/simulation at discrete time steps
signal recorded by a measurement device, e.g. sound wave
pixels of an image, grey-level or colour
3-D data measured at different X-Y-Z positions, e.g. MRIscan
. . .
Why it is useful: Memory-efficient container that provides fastnumerical operations.
In [1]: L = range(1000)
In [2]: %timeit [i**2 for i in L]1000 loops, best of 3: 403 usper loop
In [3]: a = np.arange(1000)
In [4]: %timeit a**2100000 loops, best of 3: 12.7 us perloop
NumPy Reference documentation
On the web: http://docs.scipy.org/
Interactive help: