Input:
Python variable scope error I've been programming for many years, and recently started learning Python. The following code works as expected in both python 2.5 and 3.0 (on OS X if that matters):
a, b, c = (1, 2, 3)

print(a, b, c)

def test():
    print(a)
    print(b)
    print(c)    # (A)
    #c+=1       # (B)
test()

However, when I uncomment line (B), I get an UnboundLocalError: 'c' not assigned at line (A). The values of a and b are printed correctly. This has me completely baffled for two reasons:

Why is there a runtime error thrown at line (A) because of a later statement on line (B)?
Why are variables a and b printed as expected, while c raises an error?

The only explanation I can come up with is that a local variable c is created by the assignment c+=1, which takes precedent over the "global" variable c even before the local variable is created. Of course, it doesn't make sense for a variable to "steal" scope before it exists.
Could someone please explain this behavior?

Output:
Python treats variables in functions differently depending on whether you assign values to them from within the function or not.  If you assign any value to a variable, it is treated by default as a local variable.  Therefore, when you uncomment the line, you are attempting to reference a local variable before any value has been assigned to it.
If you want the variable c to refer to the global c put
global c

as the first line of the function.
As of python 3, there is now
nonlocal c

that you can use to refer to the nearest enclosing (not necessarily global) scope.


Input:
What is the best way to call a Python script from another Python script? I have a script named test1.py which is not in a module. It just has code that should execute when the script itself is run. There are no functions, classes, methods, etc. I have another script which runs as a service. I want to call test1.py from the script running as a service.
For example:
File test1.py
print "I am a test"
print "see! I do nothing productive."

File service.py
# Lots of stuff here
test1.py # do whatever is in test1.py

I'm aware of one method which is opening the file, reading the contents, and basically eval'ing it. I'm assuming there's a better way of doing this. Or at least I hope so.

Output:
The usual way to do this is something like the following.
test1.py
def some_func():
    print 'in test 1, unproductive'

if __name__ == '__main__':
    # test1.py executed as script
    # do something
    some_func()

service.py
import test1

def service_func():
    print 'service func'

if __name__ == '__main__':
    # service.py executed as script
    # do something
    service_func()
    test1.some_func()


Input:
Easiest way to rename a model using Django/South? I've been hunting for an answer to this on South's site, Google, and SO, but couldn't find a simple way to do this.
I want to rename a Django model using South.
Say you have the following:
class Foo(models.Model):
    name = models.CharField()

class FooTwo(models.Model):
    name = models.CharField()
    foo = models.ForeignKey(Foo)

and you want to convert Foo to Bar, namely
class Bar(models.Model):
    name = models.CharField()

class FooTwo(models.Model):
    name = models.CharField()
    foo = models.ForeignKey(Bar)

To keep it simple, I'm just trying to change the name from Foo to Bar, but ignore the foo member in FooTwo for now.
What's the easiest way to do this using South?

I could probably do a data migration, but that seems pretty involved.
Write a custom migration, e.g. db.rename_table('city_citystate', 'geo_citystate'), but I'm not sure how to fix the foreign key in this case.
An easier way that you know?

Output:
To answer your first question, the simple model/table rename is pretty straightforward. Run the command:
./manage.py schemamigration yourapp rename_foo_to_bar --empty

(Update 2: try --auto instead of --empty to avoid the warning below.  Thanks to @KFB for the tip.)
If you're using an older version of south, you'll need startmigration instead of schemamigration.
Then manually edit the migration file to look like this:
class Migration(SchemaMigration):

    def forwards(self, orm):
        db.rename_table('yourapp_foo', 'yourapp_bar')

    def backwards(self, orm):
        db.rename_table('yourapp_bar','yourapp_foo')

You can accomplish this more simply using the db_table Meta option in your model class.  But every time you do that, you increase the legacy weight of your codebase -- having class names differ from table names makes your code harder to understand and maintain.  I fully support doing simple refactorings like this for the sake of clarity.
(update) I just tried this in production, and got a strange warning when I went to apply the migration.  It said:

The following content types are stale and need to be deleted:

    yourapp | foo

Any objects related to these content types by a foreign key will also
be deleted. Are you sure you want to delete these content types?
If you're unsure, answer 'no'.

I answered "no" and everything seemed to be fine.


Input:
In Python, how do I iterate over a dictionary in sorted order? There's an existing function that ends in:
return dict.iteritems()

that returns an unsorted iterator for a given dictionary. I would like to return an iterator that goes through the items in sorted order. How do I do that?

Output:
Haven't tested this very extensively, but works in Python 2.5.2.
>>> d = {"x":2, "h":15, "a":2222}
>>> it = iter(sorted(d.iteritems()))
>>> it.next()
('a', 2222)
>>> it.next()
('h', 15)
>>> it.next()
('x', 2)
>>>


Input:
Is there a way to use PhantomJS in Python? I want to use PhantomJS in Python. I googled this problem but couldn't find proper solutions.
I find os.popen()  may be a good choice. But I couldn't pass some arguments to it.
Using subprocess.Popen() may be a proper solution for now. I want to know whether there's a better solution or not.
Is there a way to use PhantomJS in Python?

Output:
The easiest way to use PhantomJS in python is via Selenium. The simplest installation method is

Install NodeJS
Using Node's package manager install phantomjs: npm -g install phantomjs
install selenium (in your virtualenv, if you are using that)

After installation, you may use phantom as simple as:
from selenium import webdriver

driver = webdriver.PhantomJS() # or add to your PATH
driver.set_window_size(1024, 768) # optional
driver.get('https://google.com/')
driver.save_screenshot('screen.png') # save a screenshot to disk
sbtn = driver.find_element_by_css_selector('button.gbqfba')
sbtn.click()

If your system path environment variable isn't set correctly, you'll need to specify the exact path as an argument to webdriver.PhantomJS().  Replace this:
driver = webdriver.PhantomJS() # or add to your PATH

... with the following:
driver = webdriver.PhantomJS(executable_path='/usr/local/lib/node_modules/phantomjs/lib/phantom/bin/phantomjs')

References:

http://selenium-python.readthedocs.org/en/latest/api.html
How do I set a proxy for phantomjs/ghostdriver in python webdriver?
http://python.dzone.com/articles/python-testing-phantomjs


Input:
Convert a number range to another range, maintaining ratio I'm trying to convert one range of numbers to another, maintaining ratio. Maths is not my strong point.
I have an image file where point values may range from -16000.00 to 16000.00 though the typical range may be much less. What I want to do is compress these values into the integer range 0-100, where 0 is the value of the smallest point, and 100 is the value of the largest. All points in between should keep a relative ratio even though some precision is being lost I'd like to do this in python but even a general algorithm should suffice. I'd prefer an algorithm where the min/max or either range can be adjusted (ie, the second range could be -50 to 800 instead of 0 to 100).

Output:
NewValue = (((OldValue - OldMin) * (NewMax - NewMin)) / (OldMax - OldMin)) + NewMin

Or a little more readable:
OldRange = (OldMax - OldMin)
NewRange = (NewMax - NewMin)
NewValue = (((OldValue - OldMin) * NewRange) / OldRange) + NewMin

Or if you want to protect for the case where the old range is 0 (OldMin = OldMax):
OldRange = (OldMax - OldMin)
if (OldRange == 0)
    NewValue = NewMin
else
{
    NewRange = (NewMax - NewMin)
    NewValue = (((OldValue - OldMin) * NewRange) / OldRange) + NewMin
}

Note that in this case we're forced to pick one of the possible new range values arbitrarily. Depending on context, sensible choices could be: NewMin (see sample), NewMax or (NewMin + NewMax) / 2


Input: How to start a background process in Python? I'm trying to port a shell script to the much more readable python version. The original shell script starts several processes (utilities, monitors, etc.) in the background with "&". How can I achieve the same effect in python? I'd like these processes not to die when the python scripts complete. I am sure it's related to the concept of a daemon somehow, but I couldn't find how to do this easily.

Output:
While jkp's solution works, the newer way of doing things (and the way the documentation recommends) is to use the subprocess module. For simple commands its equivalent, but it offers more options if you want to do something complicated.
Example for your case:
import subprocess
subprocess.Popen(["rm","-r","some.file"])

This should run rm -r somefile in the background.  But be wary: subprocess.Popen() only runs a process in the background if nothing in the python script depends on the output of the command being run:
For example, the following command will not run in the background:
import subprocess
ls_output=subprocess.Popen(["ls", "-a"], stdout=subprocess.PIPE)

See  the documentation here.
Also, a point of clarification: "Background" purely a shell concept: what you probably want is to spawn a new process. I've used "background" here to refer to shell-background-like behavior, but don't mistake this for the process actually being in the background.


Input:
Why doesn't os.path.join() work in this case? The below code will not join, when debugged the command does not store the whole path but just the last entry.
os.path.join('/home/build/test/sandboxes/', todaystr, '/new_sandbox/')

When I test this it only stores the /new_sandbox/ part of the code.

Output:
The latter strings shouldn't start with a slash. If they start with a slash, then they're considered an "absolute path" and everything before them is discarded.
Quoting the Python docs for os.path.join:

If any component is an absolute path, all previous components (on Windows, including the previous drive letter, if there was one) are thrown away, and joining continues.


Input:
Python setup.py develop vs install Two options in setup.py develop and install are confusing me. According to this site, using develop creates a special link to site-packages directory.
People have suggested that I use python setup.py install for a fresh installation and python setup.py develop after any changes have been made to the setup file.
Can anyone shed some light on the usage of these commands?

Output:
python setup.py install is used to install (typically third party) packages that you're not going to be developing/editing/debugging yourself.
For your own stuff, you want to get your package installed and then be able to frequently edit your code and not have to re-install your packageâthis is exactly what python setup.py develop does: installs the package (typically just a source folder) in a way that allows you to conveniently edit your code after its installed to the (virtual) environment and have the changes take effect immediately.


Input:
Is there a "not equal" operator in Python? How would you say does not equal?
Like
if hi == hi:
    print "hi"
elif hi (does not equal) bye:
    print "no hi"

Is there a replacement for == that means "not equal"?

Output:
Use != (preferred) or <> (deprecated). See comparison operators. For comparing object identities, you can use the keyword is and its negation is not.
e.g.
1 == 1 # true
1 != 1 # false
1 <> 1 # false
[] is [] # false (distinct objects)
a = b = []; a is b # true (same object)


Input:
How to perform OR condition in django queryset? I want to write a Django query equivalent to this SQL query:
SELECT * from user where income >= 5000 or income is NULL.

How to construct the Djagno queryset filter?
User.objects.filter(income__gte=5000, income=0)

This doesn't work, because it ANDs the filters. I want to OR the filters to get union of individual querysets.

Output:
from django.db.models import Q
User.objects.filter(Q(income__gte=5000) | Q(income__isnull=True))

via Documentation


Input:
Sort a list by multiple attributes? I have a list of lists:
[[12, 'tall', 'blue', 1],
[2, 'short', 'red', 9],
[4, 'tall', 'blue', 13]]

If I wanted to sort by one element, say the tall/short element, I could do it via s = sorted(s, key = itemgetter(1)).
If I wanted to sort by both tall/short and colour, I could do the sort twice, once for each element, but is there a quicker way?

Output:
A key can be a function that returns a tuple:
s = sorted(s, key = lambda x: (x[1], x[2]))

Or you can achieve the same using itemgetter:
import operator
s = sorted(s, key = operator.itemgetter(1, 2))

And notice that here you can use sort instead of using sorted and then reassigning:
s.sort(key = operator.itemgetter(1, 2))


Input:
Is Python strongly typed? I've come across links that say Python is a strongly typed language.
However, I thought in strongly typed languages you couldn't do this :
bob = 1
bob = "bob"

I thought a strongly typed language didn't accept type-changing at run-time. Maybe I've got a wrong (or too simplist) definition of strong/weak types.
So, is Python a strongly or weakly typed language?

Output:
Python is strongly, dynamically typed.

Strong typing means that the type of a value doesn't suddenly change. A string containing only digits doesn't magically become a number, as may happen in Perl. Every change of type requires an explicit conversion.
Dynamic typing means that runtime objects (values) have a type, as opposed to static typing where variables have a type.

As for your example
bob = 1
bob = "bob"

This works because the variable does not have a type; it can name any object. After bob=1, you'll find that type(bob) returns int, but after bob="bob", it returns str. (Note that type is a regular function, so it evaluates its argument, then returns the type of the value.)
Contrast this with older dialects of C, which were weakly, statically typed, so that pointers and integers were pretty much interchangeable. (Modern ISO C requires conversions in many cases, but my compiler is still lenient about this by default.)
I must add that the strong vs. weak typing is more of a continuum than a boolean choice. C++ has stronger typing than C (more conversions required), but the type system can be subverted by using pointer casts.
The strength of the type system in a dynamic language such as Python is really determined by how its primitives and library functions respond to different types. E.g., + is overloaded so that it works on two numbers or two strings, but not a string and an number. This is a design choice made when + was implemented, but not really a necessity following from the language's semantics. In fact, when you overload + on a custom type, you can make it implicitly convert anything to a number:
def to_number(x):
    """Try to convert x to a number."""
    if x is None:
        return 0
    # more special cases here
    else:
        return float(x)  # works for numbers and strings

class Foo(object):
    def __add__(self, other):
        other = to_number(other)
        # now do the addition

(The only language that I know that is completely strongly typed, aka strictly typed, is Haskell, where types are entirely disjoint and only a controlled form of overloading is possible via type classes.)


Input:
Python string.join(list) on object array rather than string array In Python, I can do:
>>> list = ['a', 'b', 'c']
>>> ', '.join(list)
'a, b, c'

Is there any easy way to do the same when I have a list of objects?
>>> class Obj:
...     def __str__(self):
...         return 'name'
...
>>> list = [Obj(), Obj(), Obj()]
>>> ', '.join(list)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: sequence item 0: expected string, instance found

Or do I have to resort to a for loop?

Output:
You could use a list comprehension or a generator expression instead:
', '.join([str(x) for x in list])  # list comprehension
', '.join(str(x) for x in list)    # generator expression


Input:
Python multiprocessing pool.map for multiple arguments In the Python multiprocessing library, is there a variant of pool.map which support multiple arguments?
text = "test"
def harvester(text, case):
    X = case[0]
    return text+ str(X)

if __name__ == '__main__':
    pool = multiprocessing.Pool(processes=6)
    case = RAW_DATASET
    pool.map(harvester(text,case),case, 1)
    pool.close()
    pool.join()

Output:
is there a variant of pool.map which support multiple arguments?

Python 3.3 includes pool.starmap() method:
#!/usr/bin/env python3
from functools import partial
from itertools import repeat
from multiprocessing import Pool, freeze_support

def func(a, b):
    return a + b

def main():
    a_args = [1,2,3]
    second_arg = 1
    with Pool() as pool:
        L = pool.starmap(func, [(1, 1), (2, 1), (3, 1)])
        M = pool.starmap(func, zip(a_args, repeat(second_arg)))
        N = pool.map(partial(func, b=second_arg), a_args)
        assert L == M == N

if __name__=="__main__":
    freeze_support()
    main()

For older versions:
#!/usr/bin/env python2
import itertools
from multiprocessing import Pool, freeze_support

def func(a, b):
    print a, b

def func_star(a_b):
    """Convert `f([1,2])` to `f(1,2)` call."""
    return func(*a_b)

def main():
    pool = Pool()
    a_args = [1,2,3]
    second_arg = 1
    pool.map(func_star, itertools.izip(a_args, itertools.repeat(second_arg)))

if __name__=="__main__":
    freeze_support()
    main()

Output
1 1
2 1
3 1

Notice how itertools.izip() and itertools.repeat() are used here.
Due to the bug mentioned by @unutbu you can't use functools.partial() or similar capabilities on Python 2.6, so the simple wrapper function func_star() should be defined explicitly. See also the workaround suggested by uptimebox.


Input:
Why return NotImplemented instead of raising NotImplementedError Python has a singleton called NotImplemented.
Why would someone want to ever return NotImplemented instead of raising the NotImplementedError exception? Won't it just make it harder to find bugs, such as code that executes invalid methods?

Output:
It's because __lt__() and related comparison methods are quite commonly used indirectly in list sorts and such. Sometimes the algorithm will choose to try another way or pick a default winner. Raising an exception would break out of the sort unless caught, whereas NotImplemented doesn't get raised and can be used in further tests.
http://jcalderone.livejournal.com/32837.html
To summarise that link:

"NotImplemented signals to the runtime that it should ask someone else to satisfy the operation. In the expression a == b, if a.__eq__(b) returns NotImplemented, then Python tries b.__eq__(a). If b knows enough to return True or False, then the expression can succeed. If it doesn't, then the runtime will fall back to the built-in behavior (which is based on identity for == and !=)."


Input:
How to download large file in python with requests.py? Requests is a really nice library. I'd like to use it for download big files (>1GB).
The problem is it's not possible to keep whole file in memory I need to read it in chunks. And this is a problem with the following code
import requests

def DownloadFile(url)
    local_filename = url.split('/')[-1]
    r = requests.get(url)
    f = open(local_filename, 'wb')
    for chunk in r.iter_content(chunk_size=512 * 1024):
        if chunk: # filter out keep-alive new chunks
            f.write(chunk)
    f.close()
    return

By some reason it doesn't work this way. It still loads response into memory before save it to a file.

Output:
I figured out what should be changed. The trick was to set stream = True in the get() method.
After this python process stopped to suck memory (stays around 30kb regardless size of the download file).
Thank you @danodonovan for you syntax I use it here:
def download_file(url):
    local_filename = url.split('/')[-1]
    # NOTE the stream=True parameter
    r = requests.get(url, stream=True)
    with open(local_filename, 'wb') as f:
        for chunk in r.iter_content(chunk_size=1024):
            if chunk: # filter out keep-alive new chunks
                f.write(chunk)
                #f.flush() commented by recommendation from J.F.Sebastian
    return local_filename

See http://docs.python-requests.org/en/latest/user/advanced/#body-content-workflow for further reference.


Input:
Iterate a list with indexes in Python I could swear I've seen the function (or method) that takes a list, like this [3, 7, 19] and makes it into iterable list of tuples, like so: [(0,3), (1,7), (2,19)] to use it instead of:
for i in range(len(name_of_list)):
    name_of_list[i] = something

but I can't remember the name and googling "iterate list" gets nothing.

Output:
>>> a = [3,4,5,6]
>>> for i, val in enumerate(a):
...     print i, val
...
0 3
1 4
2 5
3 6
>>>


Input:
How to hide output of subprocess in Python 2.7 I'm using eSpeak on Ubuntu and have a Python 2.7 script that prints and speaks a message:
import subprocess
text = 'Hello World.'
print text
subprocess.call(['espeak', text])

eSpeak produces the desired sounds, but clutters the shell with some errors (ALSA lib..., no socket connect) so i cannot easily read what was printed earlier. Exit code is 0.
Unfortunately there is no documented option to turn off its verbosity, so I'm looking for a way to only visually silence it and keep the open shell clean for further interaction.
How can I do this?

Output:
Redirect the output to DEVNULL:
import os
import subprocess

FNULL = open(os.devnull, 'w')
retcode = subprocess.call(['echo', 'foo'], stdout=FNULL, stderr=subprocess.STDOUT)

It is effectively the same as running this shell command:
retcode = os.system("echo 'foo' &> /dev/null")


Input:
Compact way of writing (a + b == c or a + c == b or b + c == a) Is there a more compact or pythonic way to write the boolean expression
a + b == c or a + c == b or b + c == a

I came up with
a + b + c in (2*a, 2*b, 2*c)

but that is a little strange.

Output:
If we look at the Zen of Python, emphasis mine:

The Zen of Python, by Tim Peters
Beautiful is better than ugly.
  Explicit is better than implicit.
Simple is better than complex.
  Complex is better than complicated.
  Flat is better than nested.
  Sparse is better than dense.
Readability counts.
  Special cases aren't special enough to break the rules.
  Although practicality beats purity.
  Errors should never pass silently.
  Unless explicitly silenced.
  In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
  Although that way may not be obvious at first unless you're Dutch.
  Now is better than never.
  Although never is often better than right now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
  Namespaces are one honking great idea -- let's do more of those!

The most Pythonic solution is the one that is clearest, simplest, and easiest to explain:
a + b == c or a + c == b or b + c == a

Even better, you don't even need to know Python to understand this code! It's that easy. This is, without reservation, the best solution. Anything else is intellectual masturbation.
Furthermore, this is likely the best performing solution as well, as it is the only one out of all the proposals that short circuits. If a + b == c, only a single addition and comparison is done.


Input:
Python Infinity - Any caveats? So Python has positive and negative infinity:
float("inf"), float("-inf")

This just seems like the type of feature that has to have some caveat. Is there anything I should be aware of?

Output:
You can still get not-a-number (NaN) values from simple arithmetic involving inf:
>>> 0 * float("inf")
nan

Note that you will normally not get an inf value through usual arithmetic calculations:
>>> 2.0**2
4.0
>>> _**2
16.0
>>> _**2
256.0
>>> _**2
65536.0
>>> _**2
4294967296.0
>>> _**2
1.8446744073709552e+19
>>> _**2
3.4028236692093846e+38
>>> _**2
1.157920892373162e+77
>>> _**2
1.3407807929942597e+154
>>> _**2
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
OverflowError: (34, 'Numerical result out of range')

The inf value is considered a very special value with unusual semantics, so it's better to know about an OverflowError straight away through an exception, rather than having an inf value silently injected into your calculations.


Input:
find nearest value in numpy array is there a numpy-thonic way, e.g. function, to find the 'nearest value' in an array?
example:
np.find_nearest( array, value )

Output:
import numpy as np
def find_nearest(array,value):
    idx = (np.abs(array-value)).argmin()
    return array[idx]

array = np.random.random(10)
print(array)
# [ 0.21069679  0.61290182  0.63425412  0.84635244  0.91599191  0.00213826
#   0.17104965  0.56874386  0.57319379  0.28719469]

value = 0.5

print(find_nearest(array, value))
# 0.568743859261


Input:
Case insensitive Python regular expression without re.compile In Python, I can compile a regular expression to be case-insensitive using re.compile:
>>> s = 'TeSt'
>>> casesensitive = re.compile('test')
>>> ignorecase = re.compile('test', re.IGNORECASE)
>>>
>>> print casesensitive.match(s)
None
>>> print ignorecase.match(s)
<_sre.SRE_Match object at 0x02F0B608>

Is there a way to do the same, but without using re.compile. I can't find anything like Perl's i suffix (e.g. m/test/i) in the documentation.

Output:
Pass re.IGNORECASE to the flags param of search, match, or sub:
re.search('test', 'TeSt', re.IGNORECASE)
re.match('test', 'TeSt', re.IGNORECASE)
re.sub('test', 'xxxx', 'Testing', flags=re.IGNORECASE)


Input:
Saving utf-8 texts in json.dumps as UTF8, not as \u escape sequence sample code:
>>> import json
>>> json_string = json.dumps("××¨× ×¦×§××")
>>> print json_string
"\u05d1\u05e8\u05d9 \u05e6\u05e7\u05dc\u05d4"

The problem: it's not human readable. My (smart) users want to verify or even edit text files with JSON dumps. (and i'd rather not use XML)
Is there a way to serialize objects into utf-8 json string (instead of  \uXXXX ) ?
this doesn't help:
>>> output = json_string.decode('string-escape')
"\u05d1\u05e8\u05d9 \u05e6\u05e7\u05dc\u05d4"

this works, but if any sub-objects is a python-unicode and not utf-8, it'll dump garbage:
>>> #### ok:
>>> s= json.dumps( "××¨× ×¦×§××", ensure_ascii=False)
>>> print json.loads(s)
××¨× ×¦×§××

>>> #### NOT ok:
>>> d={ 1: "××¨× ×¦×§××", 2: u"××¨× ×¦×§××" }
>>> print d
{1: '\xd7\x91\xd7\xa8\xd7\x99 \xd7\xa6\xd7\xa7\xd7\x9c\xd7\x94',
 2: u'\xd7\x91\xd7\xa8\xd7\x99 \xd7\xa6\xd7\xa7\xd7\x9c\xd7\x94'}
>>> s = json.dumps( d, ensure_ascii=False, encoding='utf8')
>>> print json.loads(s)['1']
××¨× ×¦×§××
>>> print json.loads(s)['2']
ÃÃÂ¨Ã ÃÂ¦ÃÂ§ÃÃ

i searched the json.dumps documentation but couldn't find something useful.
Edit - Solution(?):
i'll try to sum up the comments and answers by Martijn Pieters:
(edit: 2nd thought after @Sebastian's comment and about a year later)

there might be no is a built-in solution in json.dumps.
i'll have to convert all strings to UTF8 Unicode the object before it's being JSON-ed.
i'll use Mark's function that converts strings recuresively in a nested object
the example I gave depends too much on my computer & IDE environment, and doesn't run the same on all computers.

Thank you everybody :)

Output:
Use the ensure_ascii=False switch to json.dumps(), then encode the value to UTF-8 manually:
>>> json_string = json.dumps(u"××¨× ×¦×§××", ensure_ascii=False).encode('utf8')
>>> json_string
'"\xd7\x91\xd7\xa8\xd7\x99 \xd7\xa6\xd7\xa7\xd7\x9c\xd7\x94"'
>>> print json_string
"××¨× ×¦×§××"

If you are writing this to a file, you can use io.open() instead of open() to produce a file object that encodes Unicode values for you as you write, then use json.dump() instead to write to that file:
with io.open('filename', 'w', encoding='utf8') as json_file:
    json.dump(u"××¨× ×¦×§××", json_file, ensure_ascii=False)

In Python 3, the built-in open() is an alias for io.open(). Do note that there is a bug in the json module where the ensure_ascii=False flag can produce a mix of unicode and str objects. The workaround for Python 2 then is:
with io.open('filename', 'w', encoding='utf8') as json_file:
    data = json.dumps(u"××¨× ×¦×§××", ensure_ascii=False)
    # unicode(data) auto-decodes data to unicode if str
    json_file.write(unicode(data))

If you are passing in byte strings (type str in Python 2, bytes in Python 3) encoded to UTF-8, make sure to also set the encoding keyword:
>>> d={ 1: "××¨× ×¦×§××", 2: u"××¨× ×¦×§××" }
>>> d
{1: '\xd7\x91\xd7\xa8\xd7\x99 \xd7\xa6\xd7\xa7\xd7\x9c\xd7\x94', 2: u'\u05d1\u05e8\u05d9 \u05e6\u05e7\u05dc\u05d4'}

>>> s=json.dumps(d, ensure_ascii=False, encoding='utf8')
>>> s
u'{"1": "\u05d1\u05e8\u05d9 \u05e6\u05e7\u05dc\u05d4", "2": "\u05d1\u05e8\u05d9 \u05e6\u05e7\u05dc\u05d4"}'
>>> json.loads(s)['1']
u'\u05d1\u05e8\u05d9 \u05e6\u05e7\u05dc\u05d4'
>>> json.loads(s)['2']
u'\u05d1\u05e8\u05d9 \u05e6\u05e7\u05dc\u05d4'
>>> print json.loads(s)['1']
××¨× ×¦×§××
>>> print json.loads(s)['2']
××¨× ×¦×§××

Note that your second sample is not valid Unicode; you gave it UTF-8 bytes as a unicode literal, that would never work:
>>> s = u'\xd7\x91\xd7\xa8\xd7\x99 \xd7\xa6\xd7\xa7\xd7\x9c\xd7\x94'
>>> print s
ÃÃÂ¨Ã ÃÂ¦ÃÂ§ÃÃ
>>> print s.encode('latin1').decode('utf8')
××¨× ×¦×§××

Only when I encoded that string to Latin 1 (whose unicode codepoints map one-to-one to bytes) then decode as UTF-8 do you see the expected output. That has nothing to do with JSON and everything to do with that you use the wrong input. The result is called a Mojibake.
If you got that Unicode value from a string literal, it was decoded using the wrong codec. It could be your terminal is mis-configured, or that your text editor saved your source code using a different codec than what you told Python to read the file with. Or you sourced it from a library that applied the wrong codec. This all has nothing to do with the JSON library.


Input:
How to delete items from a dictionary while iterating over it? Is it legitimate to delete items from a dictionary in Python while iterating over it?
For example:
for k, v in mydict.iteritems():
   if k == val:
     del mydict[k]

The idea is to remove elements that don't meet a certain condition from the dictionary, instead of creating a new dictionary that's a subset of the one being iterated over.
Is this a good solution? Are there more elegant/efficient ways?

Output:
A simple test in the console shows you cannot modify a dictionary while iterating over it:
>>> mydict = {'one': 1, 'two': 2, 'three': 3, 'four': 4}
>>> for k, v in mydict.iteritems():
...    if k == 'two':
...        del mydict[k]
...
------------------------------------------------------------
Traceback (most recent call last):
  File "<ipython console>", line 1, in <module>
RuntimeError: dictionary changed size during iteration

As stated in delnan's answer, deleting entries causes problems when the iterator tries to move onto the next entry. Instead, use the keys() method to get a list of the keys and work with that:
>>> for k in mydict.keys():
...    if k == 'two':
...        del mydict[k]
...
>>> mydict
{'four': 4, 'three': 3, 'one': 1}

If you need to delete based on the items value, use the items() method instead:
>>> for k, v in mydict.items():
...     if v == 3:
...         del mydict[k]
...
>>> mydict
{'four': 4, 'one': 1}


Input:
How to execute a file within the python interpreter? I'm trying to execute a file with python commands from within the interpreter.
EDIT: I'm trying to use variables and settings from that file, not to invoke a separate process.

Output:
Several ways.
From the shell
python someFile.py

From inside IDLE, hit F5.
If you're typing interactively, try this.
>>> variables= {}
>>> execfile( "someFile.py", variables )
>>> print variables # globals from the someFile module


Input:
Python dictionary: are keys() and values() always the same order? It looks like the lists returned by keys() and values() methods of a dictionary are always a 1-to-1 mapping (assuming the dictionary is not altered between calling the 2 methods).
For example:
>>> d = {'one':1, 'two': 2, 'three': 3}
>>> k, v = d.keys(), d.values()
>>> for i in range(len(k)):
    print d[k[i]] == v[i]

True
True
True

If you do not alter the dictionary between calling keys() and calling values(), is it wrong to assume the above for-loop will always print True?  I could not find any documentation confirming this.

Output:
Found this:

If items(), keys(), values(),
  iteritems(), iterkeys(), and
  itervalues() are called with no
  intervening modifications to the
  dictionary, the lists will directly
  correspond.

On 2.x documentation and 3.x documentation.


Input:
Python: split a list based on a condition? What's the best way, both aesthetically and from a performance perspective, to split a list of items into multiple lists based on a conditional? The equivalent of:
good = [x for x in mylist if x in goodvals]
bad  = [x for x in mylist if x not in goodvals]

is there a more elegant way to do this?
Update: here's the actual use case, to better explain what I'm trying to do:
# files looks like: [ ('file1.jpg', 33L, '.jpg'), ('file2.avi', 999L, '.avi'), ... ]
IMAGE_TYPES = ('.jpg','.jpeg','.gif','.bmp','.png')
images = [f for f in files if f[2].lower() in IMAGE_TYPES]
anims  = [f for f in files if f[2].lower() not in IMAGE_TYPES]

Output:
good, bad = [], []
for x in mylist:
    (bad, good)[x in goodvals].append(x)


Input:
How to use a different version of python during NPM install? Salam (means Hello) :)
I have terminal access to a VPS running centos 5.9 and default python 2.4.3 installed. I also installed python 2.7.3 via these commands: (I used make altinstall instead of make install)
wget http://www.python.org/ftp/python/2.7.3/Python-2.7.3.tgz
tar -xf Python-2.7.3.tgz
cd Python-2.7.3
./configure
make
make altinstall

then I installed node.js from source via these commands:
python2.7 ./configure
make
make install

The problem is, when I use npm install and try to install a node.js package which requires python > 2.4.3 I get this error:
gyp ERR! configure error
gyp ERR! stack Error: Python executable "python" is v2.4.3, which is not supported by gyp.
gyp ERR! stack You can pass the --python switch to point to Python >= v2.5.0 & < 3.0.0.
gyp ERR! stack     at failPythonVersion (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:125:14)
gyp ERR! stack     at /usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:114:9

how should I "pass the --python switch to point to Python >= v2.5.0"?

Output:
You can use --python option to npm like so:
npm install --python=python2.7

or set it to be used always:
npm config set python python2.7

Npm will in turn pass this option to node-gyp when needed.
(note: I'm the one who opened an issue on Github to have this included in the docs, as there were so many questions about it ;-) )


Input:
Getting method parameter names in python Given the python function:
def aMethod(arg1, arg2):
    pass

How can I extract the number and names of the arguments. Ie. given that I have a reference to func, I want the func.[something] to return ("arg1", "arg2")
The usage scenario for this is that I have a decorator, and I wish to use the method arguments in the same order that they appear for the actual function as a key. Ie. how would the decorator look that printed "a,b" when I call aMethod("a","b")

Output:
Take a look at the inspect module - this will do the inspection of the various code object properties for you.
>>> inspect.getargspec(aMethod)
(['arg1', 'arg2'], None, None, None)

The other results are the name of the *args and **kwargs variables, and the defaults provided.  ie.
>>> def foo(a,b,c=4, *arglist, **keywords): pass
>>> inspect.getargspec(foo)
(['a', 'b', 'c'], 'arglist', 'keywords', (4,))


Input:
python decorators with parameters I have a problem with the transfer of variable 'insurance_mode' by the decorator. I would do it by the following decorator statement:
 @execute_complete_reservation(True)
 def test_booking_gta_object(self):
     self.test_select_gta_object()

but unfortunately, this statement does not work. Perhaps maybe there is better way to solve this problem.
def execute_complete_reservation(test_case,insurance_mode):
    def inner_function(self,*args,**kwargs):
        self.test_create_qsf_query()
        test_case(self,*args,**kwargs)
        self.test_select_room_option()
        if insurance_mode:
            self.test_accept_insurance_crosseling()
        else:
            self.test_decline_insurance_crosseling()
        self.test_configure_pax_details()
        self.test_configure_payer_details

    return inner_function

Output:
You mean def test_booking_gta_object, right? Anyway, the syntax for decorators with arguments is a bit different - the decorator with arguments should return a function that will take a function and return another function. So it should really return a normal decorator. A bit confusing, right? What I mean is:
def decorator(argument):
    def real_decorator(function):
        def wrapper(*args, **kwargs):
            funny_stuff()
            something_with_argument(argument)
            function(*args, **kwargs)
            more_funny_stuff()
        return wrapper
     return real_decorator

Here you can read more on the subject - it's also possible to implement this using callable objects and that is also explained there.


Input:
Rename a dictionary key Is there a way to rename a dictionary key, without reassigning its value to a new name and removing the old name key; and without iterating through dict key/value?
In case of OrderedDict, do the same, while keeping that key's position.

Output:
For a regular dict, you can use:
mydict[new_key] = mydict.pop(old_key)

For an OrderedDict, I think you must build an entirely new one using a comprehension.
>>> OrderedDict(zip('123', 'abc'))
OrderedDict([('1', 'a'), ('2', 'b'), ('3', 'c')])
>>> oldkey, newkey = '2', 'potato'
>>> OrderedDict((newkey if k == oldkey else k, v) for k, v in _.viewitems())
OrderedDict([('1', 'a'), ('potato', 'b'), ('3', 'c')])

Modifying the key itself, as this question seems to be asking, is impractical because dict keys are usually immutable objects such as numbers, strings or tuples.  Instead of trying to modify the key, reassigning the value to a new key and removing the old key is how you can achieve the "rename" in python.


Input: Python + Django page redirect How do I accomplish a simple redirect (e.g. cflocation in ColdFusion, or header(location:http://) in Django)?

Output:
It's simple:
from django.http import HttpResponseRedirect

def myview(request):
    ...
    return HttpResponseRedirect("/path/")

More info in the official Django docs
Update: Django 1.0
There is apparently a better way of doing this in Django now using generic views.
Example -
from django.views.generic.simple import redirect_to

urlpatterns = patterns('',
    (r'^one/$', redirect_to, {'url': '/another/'}),

    #etc...
)

There is more in the generic views documentation.
Credit - Carles BarrobÃ©s.
Update #2: Django 1.3+
In Django 1.5 *redirect_to* no longer exists and has been replaced by RedirectView. Credit to Yonatan
from django.views.generic import RedirectView

urlpatterns = patterns('',
    (r'^one/$', RedirectView.as_view(url='/another/')),
)


Input: How to send email attachments with Python I am having problems understanding how to email an attachment using Python. I have successfully emailed simple messages with the smtplib. Could someone please explain how to send an attachment in an email. I know there are other posts online but as a Python beginner I find them hard to understand.

Output:
Here's another, adapted from here:
import smtplib
from os.path import basename
from email.mime.application import MIMEApplication
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText
from email.utils import COMMASPACE, formatdate

def send_mail(send_from, send_to, subject, text, files=None,
              server="127.0.0.1"):
    assert isinstance(send_to, list)

    msg = MIMEMultipart()
    msg['From'] = send_from
    msg['To'] = COMMASPACE.join(send_to)
    msg['Date'] = formatdate(localtime=True)
    msg['Subject'] = subject

    msg.attach(MIMEText(text))

    for f in files or []:
        with open(f, "rb") as fil:
            part = MIMEApplication(
                fil.read(),
                Name=basename(f)
            )
            part['Content-Disposition'] = 'attachment; filename="%s"' % basename(f)
            msg.attach(part)

    smtp = smtplib.SMTP(server)
    smtp.sendmail(send_from, send_to, msg.as_string())
    smtp.close()

It's much the same as the first example... But it should be easier to drop in.


Input:
Python creating a dictionary of lists I want to create a dictionary whose values are lists.  For example:
{
  1: ['1'],
  2: ['1','2'],
  3: ['2']
}

If I do:
d = dict()
a = ['1', '2']
for i in a:
    for j in range(int(i), int(i) + 2):
        d[j].append(i)

I get a KeyError, because d[...] isn't a list.  In this case, I can add the following code after the assignment of a to initialize the dictionary.
for x in range(1, 4):
    d[x] = list()

Is there a better way to do this?  Lets say I don't know the keys I am going to need until I am in the second for loop.  For example:
class relation:
    scope_list = list()
...
d = dict()
for relation in relation_list:
    for scope_item in relation.scope_list:
        d[scope_item].append(relation)

An alternative would then be replacing
d[scope_item].append(relation)

with
if d.has_key(scope_item):
    d[scope_item].append(relation)
else:
    d[scope_item] = [relation,]

What is the best way to handle this?  Ideally, appending would "just work".  Is there some way to express that I want a dictionary of empty lists, even if I don't know every key when I first create the list?

Output:
You can use defaultdict:
>>> from collections import defaultdict
>>> d = defaultdict(list)
>>> for i in a:
...   for j in range(int(i), int(i) + 2):
...     d[j].append(i)
...
>>> d
defaultdict(<type 'list'>, {1: ['1'], 2: ['1', '2'], 3: ['2']})
>>> d.items()
[(1, ['1']), (2, ['1', '2']), (3, ['2'])]


Input:
Changing the "tick frequency" on x or y axis in matplotlib? I am trying to fix how python plots my data.
Say
x = [0,5,9,10,15]

and
y = [0,1,2,3,4]

Then I would do:
matplotlib.pyplot.plot(x,y)
matplotlib.pyplot.show()

and the x axis' ticks are plotted in intervals of 5. Is there a way to make it show intervals of 1?

Output:
You could explicitly set where you want to tick marks with plt.xticks:
plt.xticks(np.arange(min(x), max(x)+1, 1.0))

For example,
import numpy as np
import matplotlib.pyplot as plt

x = [0,5,9,10,15]
y = [0,1,2,3,4]
plt.plot(x,y)
plt.xticks(np.arange(min(x), max(x)+1, 1.0))
plt.show()

(np.arange was used rather than Python's range function just in case min(x) and max(x) are floats instead of ints.)

The plt.plot (or ax.plot) function will automatically set default x and y limits. If you wish to keep those limits, and just change the stepsize of the tick marks, then you could use ax.get_xlim() to discover what limits Matplotlib has already set.
start, end = ax.get_xlim()
ax.xaxis.set_ticks(np.arange(start, end, stepsize))

The default tick formatter should do a decent job rounding the tick values to a sensible number of significant digits. However, if you wish to have more control over the format, you can define your own formatter. For example,
ax.xaxis.set_major_formatter(ticker.FormatStrFormatter('%0.1f'))

Here's a runnable example:
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.ticker as ticker

x = [0,5,9,10,15]
y = [0,1,2,3,4]
fig, ax = plt.subplots()
ax.plot(x,y)
start, end = ax.get_xlim()
ax.xaxis.set_ticks(np.arange(start, end, 0.712123))
ax.xaxis.set_major_formatter(ticker.FormatStrFormatter('%0.1f'))
plt.show()


Input:
Get the cartesian product of a series of lists in Python How can I get the Cartesian product (every possible combination of values) from a group of lists?
Input:
somelists = [
   [1, 2, 3],
   ['a', 'b'],
   [4, 5]
]

Desired output:
[(1, 'a', 4), (1, 'a', 5), (1, 'b', 4), (1, 'b', 5), (2, 'a', 4), (2, 'a', 5) ...]

Output:
In Python 2.6+
import itertools
for element in itertools.product(*somelists):
    print element

Documentation:
Python 2 - itertools.product


Input:
Why dict.get(key) instead of dict[key]? Today, I came across the dict method get which, given a key in the dictionary, returns the associated value.
For what purpose is this function useful? If I wanted to find a value associated with a key in a dictionary, I can just do dict[key], and it returns the same thing:
dictionary = {"Name": "Harry", "Age": 17}
dictionary["Name"]
dictionary.get("Name")

Output:
It allows you to provide a default value if the key is missing:
dictionary.get("bogus", None)

returns None, whereas
dictionary["bogus"]

would raise a KeyError.


Input:
How do I override __getattr__ in Python without breaking the default behavior? I want to override the __getattr__ method on a class to do something fancy but I don't want to break the default behavior.
What's the correct way to do this?

Output:
Overriding __getattr__ should be fine -- __getattr__ is only called as a last resort i.e. if there are no attributes in the instance that match the name. For instance, if you access foo.bar, then __getattr__ will only be called if foo has no attribute called bar. If the attribute is one you don't want to handle, raise AttributeError:
class Foo(object):
    def __getattr__(self, name):
        if some_predicate(name):
            # ...
        else:
            # Default behaviour
            raise AttributeError

However, unlike __getattr__, __getattribute__ will be called first (only works for new style classes i.e. those that inherit from object). In this case, you can preserve default behaviour like so:
class Foo(object):
    def __getattribute__(self, name):
        if some_predicate(name):
            # ...
        else:
            # Default behaviour
            return object.__getattribute__(self, name)

See the Python docs for more.


Input:
Configuring so that pip install can work from github We'd like to use pip with github to install private packages to our production servers.  This question concerns what needs to be in the github repo in order for the install to be successful.
Assuming the following command line (which authenticates just fine and tries to install):
pip install git+ssh://git@github.com/BlahCo/search/tree/prod_release_branch/ProductName

What needs to reside in the ProductName?  Is it the contents of what would normally be in the tar file after running setup.py with the sdist option, or is the actual tar.gz file, or something else?
I'm asking here because I've tried several variations and can't make it work.  Any help appreciated.

Output:
You need the whole python package, with a setup.py file in it.
A package named foo would be:
foo # the installable package
âââ foo
âÂ Â  âââ __init__.py
âÂ Â  âââ bar.py
âââ setup.py

And install from github like:
$ pip install git+git://github.com/myuser/foo.git@v123
or
$ pip install git+git://github.com/myuser/foo.git@newbranch

More info at http://www.pip-installer.org/en/latest/reference/pip_install.html#vcs-support


Input:
I can't install python-ldap When I run the following command:
sudo pip install python-ldap

I get this error:

In file included from Modules/LDAPObject.c:9:
Modules/errors.h:8: fatal error: lber.h: No such file or directory

Any ideas how to fix this?

Output:
The website says that python-ldap is based on OpenLDAP, so you need to have the development files (headers) in order to compile the Python module. If you're on Ubuntu, the package is called libldap2-dev.
sudo apt-get install libsasl2-dev python-dev libldap2-dev libssl-dev


Input:
What do (lambda) function closures capture in Python? Recently I started playing around with Python and I came around something peculiar in the way closures work. Consider the following code:
adders= [0,1,2,3]
for i in [0,1,2,3]:
   adders[i]=lambda a: i+a

print adders[1](3)

It builds a simple array of functions that take a single input and return that input added by a number. The functions are constructed in for loop where the iterator i runs from 0 to 3. For each of these number a lambda funciton is created which captures i and adds it to the function's input. The last line calls the second lambda function with 3 as a parameter. To my surprise the output was:
6

I expected a 4. My reasoning was: in Python everything is an object and thus every variable is essential a pointer to it. When creating the lambda closures for i, I expected it to store a pointer to the integer object currently pointed to by i. That means that when i assigned a new integer object it shouldn't effect the previously created closures. Sadly, inspecting the adders array within a debugger shows that it does. All lambda functions refer to the last value of i, 3, which results in adders1 returning 6.
Which me the following:

what does the closures capture exactly?
What is the most elegant way to convince that lambda functions to capture the current value of i and in a way that will not be affected when i changes it's value.

Output:
you may force the capture of a variable using an argument with a default value:
>>> for i in [0,1,2,3]:
...    adders[i]=lambda a,i=i: i+a  # note the dummy parameter with a default value
...
>>> print( adders[1](3) )
4

the idea is to declare a parameter (cleverly named i) and give it a default value of the variable you want to capture (the value of  i)


Input:
How do I design a class in Python? I've had some really awesome help on my previous questions for detecting paws and toes within a paw, but all these solutions only work for one measurement at a time.
Now I have data that consists off:

about 30 dogs;
each has 24 measurements (divided into several subgroups);
each measurement has at least 4 contacts (one for each paw) and

each contact is divided into 5 parts and
has several parameters, like contact time, location, total force etc.

Obviously sticking everything into one big object isn't going to cut it, so I figured I needed to use classes instead of the current slew of functions. But even though I've read Learning Python's chapter about classes, I fail to apply it to my own code (GitHub link)
I also feel like it's rather strange to process all the data every time I want to get out some information. Once I know the locations of each paw, there's no reason for me to calculate this again. Furthermore, I want to compare all the paws of the same dog to determine which contact belongs to which paw (front/hind, left/right). This would become a mess if I continue using only functions.
So now I'm looking for advice on how to create classes that will let me process my data (link to the zipped data of one dog) in a sensible fashion.

Output:
How to design a class.

Write down the words.  You started to do this.  Some people don't and wonder why they have problems.
Expand your set of words into simple statements about what these objects will be doing.  That is to say, write down the various calculations you'll be doing on these things.  Your short list of 30 dogs, 24 measurements, 4 contacts, and several "parameters" per contact is interesting, but only part of the story.  Your "locations of each paw" and "compare all the paws of the same dog to determine which contact belongs to which paw" are the next step in object design.
Underline the nouns.  Seriously.  Some folks debate the value of this, but I find that for first-time OO developers it helps.  Underline the nouns.
Review the nouns.  Generic nouns like "parameter" and "measurement" need to be replaced with specific, concrete nouns that apply to your problem in your problem domain.  Specifics help clarify the problem.  Generics simply elide details.
For each noun ("contact", "paw", "dog", etc.) write down the attributes of that noun and the actions in which that object engages.  Don't short-cut this.  Every attribute.  "Data Set contains 30 Dogs" for example is important.
For each attribute, identify if this is a relationship to a defined noun, or some other kind of "primitive" or "atomic" data like a string or a float or something irreducible.
For each action or operation, you have to identify which noun has the responsibility, and which nouns merely participate.  It's a  question of "mutability".  Some objects get updated, others don't.  Mutable objects must own total responsibility for their mutations.
At this point, you can start to transform nouns into class definitions.  Some collective nouns are lists, dictionaries, tuples, sets or namedtuples, and you don't need to do very much work.  Other classes are more complex, either because of complex derived data or because of some update/mutation which is performed.

Don't forget to test each class in isolation using unittest.
Also, there's no law that says classes must be mutable.  In your case, for example, you have almost no mutable data.  What you have is derived data, created by transformation functions from the source dataset.


Input:
No Multiline Lambda in Python: Why not? I've heard it said that multiline lambdas can't be added in Python because they would clash syntactically with the other syntax constructs in Python.  I was thinking about this on the bus today and realized I couldn't think of a single Python construct that multiline lambdas clash with.  Given that I know the language pretty well, this surprised me.
Now, I'm sure Guido had a reason for not including multiline lambdas in the language, but out of curiosity: what's a situation where including a multiline lambda would be ambiguous?  Is what I've heard true, or is there some other reason that Python doesn't allow multiline lambdas?

Output:
Guido van Rossum (the inventor of Python) answers this exact question himself in an old blog post.
Basically, he admits that it's theoretically possible, but that any proposed solution would be un-Pythonic:

"But the complexity of any proposed solution for this puzzle is immense, to me: it requires the parser (or more precisely, the lexer) to be able to switch back and forth between indent-sensitive and indent-insensitive modes, keeping a stack of previous modes and indentation level. Technically that can all be solved (there's already a stack of indentation levels that could be generalized). But none of that takes away my gut feeling that it is all an elaborate Rube Goldberg contraption."


Input:
Reusable library to get human readable version of file size? There are various snippets on the web that would give you a function to return human readable size from bytes size:
>>> human_readable(2048)
'2 kilobytes'
>>>

But is there a Python library that provides this?

Output:
Addressing the above "too small a task to require a library" issue by a straightforward implementation:
def sizeof_fmt(num, suffix='B'):
    for unit in ['','Ki','Mi','Gi','Ti','Pi','Ei','Zi']:
        if abs(num) < 1024.0:
            return "%3.1f%s%s" % (num, unit, suffix)
        num /= 1024.0
    return "%.1f%s%s" % (num, 'Yi', suffix)

Supports:

all currently known binary prefixes
negative and positive numbers
numbers larger than 1000 Yobibytes
arbitrary units (maybe you like to count in Gibibits!)

Example:
>>> sizeof_fmt(168963795964)
'157.4GiB'

by Fred Cirera


Input:
How to check if a user is logged in (how to properly use user.is_authenticated)? I am looking over this website but just can't seem to figure out how to do this as it's not working. I need to check if the current site user is logged in (authenticated), and am trying:
request.user.is_authenticated

despite being sure that the user is logged in, it returns just:
>

I'm able to do other requests (from the first section in the url above), such as:
request.user.is_active

which returns a successful response.

Output:
is_authenticated is a function. You should call it like
if request.user.is_authenticated():
    # do something if the user is authenticated

As Peter Rowell pointed out, what may be tripping you up is that in the default Django template language, you don't tack on parenthesis to call functions. So you may have seen something like this in template code:
{% if user.is_authenticated %}

However, in Python code, it is indeed a method in the User class.


Input:
How to change legend size with matplotlib.pyplot Simple question here:  I'm just trying to get the size of my legend using matplotlib.pyplot to be smaller (ie, the text to be smaller).  The code I'm using goes something like this:
plot.figure()
plot.scatter(k,sum_cf, color = 'black', label='Sum of Cause Fractions')
plot.scatter(k, data[:, 0],  color = 'b', label = 'Dis 1: cf = .6, var = .2')
plot.scatter(k, data[:, 1],  color = 'r',  label = 'Dis 2: cf = .2, var = .1')
plot.scatter(k, data[:, 2],  color = 'g', label = 'Dis 3: cf = .1, var = .01')
plot.legend(loc=2)

Output:
You can set an individual font size for the legend by adjusting the 'prop' keyword.
plot.legend(loc=2,prop={'size':6})

This takes a dictionary of keywords corresponding to matplotlib.font_manager.FontProperties properties. See the documentation for legend:

Keyword arguments:
prop: [ None | FontProperties | dict ]
    A matplotlib.font_manager.FontProperties instance. If prop is a
    dictionary, a new instance will be created with prop. If None, use
    rc settings.

It is also possible, as of version 1.2.1, to use the keyword fontsize.


Input:
What is the correct syntax for 'else if'? I'm a new Python programmer who is making the leap from 2.6.4 to 3.1.1. Everything has gone fine until I tried to use the 'else if' statement. The interpreter gives me a syntax error after the 'if' in 'else if' for a reason I can't seem to figure out.
def function(a):
    if a == '1':
        print ('1a')
    else if a == '2'
        print ('2a')
    else print ('3a')

function(input('input:'))

I'm probably missing something very simple; however, I haven't been able to find the answer on my own.

Output:
In python "else if" is spelled "elif".
Also, you need a colon after the elif and the else.
Simple answer to a simple question. I had the same problem, when I first started (in the last couple of weeks).
So your code should read:
def function(a):
    if a == '1':
        print('1a')
    elif a == '2':
        print('2a')
    else:
        print('3a')

function(input('input:'))


Input:
Convert string representation of list to list in Python I was wondering what the simplest way is to convert a string list like the following to a list:
x = u'[ "A","B","C" , " D"]'

Even in case user puts spaces in between the commas, and spaces inside of the quotes. I need to handle that as well to:
x = ["A", "B", "C", "D"]

in Python.
I know I can strip spaces with strip() and split() using the split operator and check for non alphabets. But the code was getting very kludgy. Is there a quick function that I'm not aware of?

Output:
>>> import ast
>>> x = u'[ "A","B","C" , " D"]'
>>> x = ast.literal_eval(x)
>>> x
['A', 'B', 'C', ' D']
>>> x = [n.strip() for n in x]
>>> x
['A', 'B', 'C', 'D']

ast.literal_eval:

Safely evaluate an expression node or a string containing a Python expression. The string or node provided may only consist of the following Python literal structures: strings, numbers, tuples, lists, dicts, booleans, and None.


Input:
Can't install PIL after Mac OS X 10.9 I've just updated my Mac OS to 10.9 and I discovered that some (all?) of my Python modules are not here anymore, especially the Image one.
So I try to execute sudo pip install pil, but I get this error:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/tk.h:78:11: fatal error: 'X11/Xlib.h' file not found

#      include <X11/Xlib.h>

               ^

1 error generated.

error: command 'cc' failed with exit status 1

My Xcode is up-to-date and I don't have any idea. Is it possible that PIL is not yet 10.9 compatible ?

Output:
Following worked for me:
ln -s  /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/System/Library/Frameworks/Tk.framework/Versions/8.5/Headers/X11 /usr/local/include/X11
sudo pip install pil

UPDATE:
But there is more correct solution below, provided by Will.

open your terminal and execute:
  xcode-select --install


Input:
Writing Unicode text to a text file? I'm pulling data out of a Google doc, processing it, and writing it to a file (that eventually I will paste into a Wordpress page).
It has some non-ASCII symbols. How can I convert these safely to symbols that can be used in HTML source?
Currently I'm converting everything to Unicode on the way in, joining it all together in a Python string, then doing:
import codecs
f = codecs.open('out.txt', mode="w", encoding="iso-8859-1")
f.write(all_html.encode("iso-8859-1", "replace"))

There is an encoding error on the last line:

UnicodeDecodeError: 'ascii' codec can't decode byte 0xa0 in position
  12286: ordinal not in range(128)

Partial solution:
This Python runs without an error:
row = [unicode(x.strip()) if x is not None else u'' for x in row]
all_html = row[0] + "<br/>" + row[1]
f = open('out.txt', 'w')
f.write(all_html.encode("utf-8")

But then if I open the actual text file, I see lots of symbols like:
QurâÃÃ´an

Maybe I need to write to something other than a text file?

Output:
Deal exclusively with unicode objects as much as possible by decoding things to unicode objects when you first get them and encoding them as necessary on the way out.
If your string is actually a unicode object, you'll need to convert it to a unicode-encoded string object before writing it to a file:
foo = u'Î, Ð, ×§, â Ù, à¹, ã, å¶, è, and ë§.'
f = open('test', 'w')
f.write(foo.encode('utf8'))
f.close()

When you read that file again, you'll get a unicode-encoded string that you can decode to a unicode object:
f = file('test', 'r')
print f.read().decode('utf8')


Input: How to install Python MySQLdb module using pip? How can I install the MySQLdb module for Python using pip?

Output:
It's easy to do, but hard to remember the correct spelling:
pip install MySQL-python

Note: Some dependencies might have to be in place when running the above command. Some hints on how to install these on various platforms:
Ubuntu 14, Ubuntu 16, Debian 8.6 (jessie)
sudo apt-get install python-pip python-dev libmysqlclient-dev

Fedora 24:
sudo dnf install python python-devel mysql-devel redhat-rpm-config gcc


Input:
Sleeping in a batch file When writing a batch file to automate something on a Windows box, I've needed to pause its execution for several seconds (usually in a test/wait loop, waiting for a process to start). At the time, the best solution I could find uses ping (I kid you not) to achieve the desired effect. I've found a better write-up of it here, which describes a callable "wait.bat", implemented as follows:
@ping 127.0.0.1 -n 2 -w 1000 > nul
@ping 127.0.0.1 -n %1% -w 1000> nul

You can then include calls to wait.bat in your own batch file, passing in the number of seconds to sleep.
Apparently the Windows 2003 Resource Kit provides a Unix-like sleep command (at last!). In the meantime, for those of us still using Windows XP, Windows 2000 or (sadly) Windows NT, is there a better way?
I modified the sleep.py script in the accepted answer, so that it defaults to one second if no arguments are passed on the command line:
import time, sys

time.sleep(float(sys.argv[1]) if len(sys.argv) > 1 else 1)

Output:
The timeout command is available from Vista onwards.
c:\> timeout /?

TIMEOUT [/T] timeout [/NOBREAK]

Description:
    This utility accepts a timeout parameter to wait for the specified
    time period (in seconds) or until any key is pressed. It also
    accepts a parameter to ignore the key press.

Parameter List:
    /T        timeout       Specifies the number of seconds to wait.
                            Valid range is -1 to 99999 seconds.

    /NOBREAK                Ignore key presses and wait specified time.

    /?                      Displays this help message.

NOTE: A timeout value of -1 means to wait indefinitely for a key press.

Examples:
    TIMEOUT /?
    TIMEOUT /T 10
    TIMEOUT /T 300 /NOBREAK
    TIMEOUT /T -1

Note: Does not work with input redirection - trivial example:
C:\>echo 1 | timeout /t 1 /nobreak
ERROR: Input redirection is not supported, exiting the process immediately.


Input:
Why are Python's arrays slow? I expected array.array to be faster than lists, as arrays seem to be unboxed.
However, I get the following result:
In [1]: import array

In [2]: L = list(range(100000000))

In [3]: A = array.array('l', range(100000000))

In [4]: %timeit sum(L)
1 loop, best of 3: 667 ms per loop

In [5]: %timeit sum(A)
1 loop, best of 3: 1.41 s per loop

In [6]: %timeit sum(L)
1 loop, best of 3: 627 ms per loop

In [7]: %timeit sum(A)
1 loop, best of 3: 1.39 s per loop

What could be the cause of such a difference?

Output:
The storage is "unboxed", but every time you access an element Python has to "box" it (embed it in a regular Python object) in order to do anything with it.  For example, your sum(A) iterates over the array, and boxes each integer, one at a time, in a regular Python int object.  That costs time.  In your sum(L), all the boxing was done at the time the list was created.
So, in the end, an array is generally slower, but requires substantially less memory.

Here's the relevant code from a recent version of Python 3, but the same basic ideas apply to all CPython implementations since Python was first released.
Here's the code to access a list item:
PyObject *
PyList_GetItem(PyObject *op, Py_ssize_t i)
{
    /* error checking omitted */
    return ((PyListObject *)op) -> ob_item[i];
}

There's very little to it:  somelist[i] just returns the i'th object in the list (and all Python objects in CPython are pointers to a struct whose initial segment conforms to the layout of a struct PyObject).
And here's the __getitem__ implementation for an array with type code l:
static PyObject *
l_getitem(arrayobject *ap, Py_ssize_t i)
{
    return PyLong_FromLong(((long *)ap->ob_item)[i]);
}

The raw memory is treated as a vector of platform-native C long integers; the i'th C long is read up; and then PyLong_FromLong() is called to wrap ("box") the native C long in a Python long object (which, in Python 3, which eliminates Python 2's distinction between int and long, is actually shown as type int).
This boxing has to allocate new memory for a Python int object, and spray the native C long's bits into it.  In the context of the original example, this object's lifetime is very brief (just long enough for sum() to add the contents into a running total), and then more time is required to deallocate the new int object.
This is where the speed difference comes from, always has come from, and always will come from in the CPython implementation.


Input:
Binary search (bisection) in Python Is there a library function that performs binary search on a list/tuple and return the position of the item if found and 'False' (-1, None, etc.) if not?
I found the functions bisect_left/right in the bisect module, but they still return a position even if the item is not in the list. That's perfectly fine for their intended usage, but I just want to know if an item is in the list or not (don't want to insert anything).
I thought of using bisect_left and then checking if the item at that position is equal to what I'm searching, but that seems cumbersome (and I also need to do bounds checking if the number can be larger than the largest number in my list). If there is a nicer method I'd like to know about it.
Edit To clarify what I need this for: I'm aware that a dictionary would be very well suited for this, but I'm trying to keep the memory consumption as low as possible. My intended usage would be a sort of double-way look-up table. I have in the table a list of values and I need to be able to access the values based on their index. And also I want to be able to find the index of a particular value or None if the value is not in the list.
Using a dictionary for this would be the fastest way, but would (approximately) double the memory requirements.
I was asking this question thinking that I may have overlooked something in the Python libraries. It seems I'll have to write my own code, as Moe suggested.

Output:
from bisect import bisect_left

def binary_search(a, x, lo=0, hi=None):   # can't use a to specify default for hi
    hi = hi if hi is not None else len(a) # hi defaults to len(a)
    pos = bisect_left(a,x,lo,hi)          # find insertion position
    return (pos if pos != hi and a[pos] == x else -1) # don't walk off the end


Input:
Installing SetupTools on 64-bit Windows I'm running Python 2.7 on Windows 7 64-bit, and when I run the installer for setuptools it tells me that Python 2.7 is not installed.  The specific error message is:
`Python Version 2.7 required which was not found in the registry`

My installed version of Python is:
`Python 2.7 (r27:82525, Jul  4 2010, 07:43:08) [MSC v.1500 64 bit (AMD64)] on win32`

I'm looking at the setuptools site and it doesn't mention any installers for 64-bit Windows.  Have I missed something or do I have to install this from source?

Output:
Problem: you have 64-bit Python, and a 32-bit installer. This will cause problems for extension modules.
The reasons why the installer doesn't finds Python is the transparent 32-bit emulation from Windows 7. 64-bit and 32-bit programs will write to different parts of the Windows registry.
64-bit: HKLM|HKCU\SOFTWARE\
32-bit: HKLM|HKCU\SOFTWARE\wow6432node\.
This means that the 64-bit Python installer writes to HKLM\SOFTWARE\Python, but the 32-bit setuptools installer looks at HKLM\SOFTWARE\wow6432node\Python (this is handled by windows automatically, programs don't notice). This is expected behavior and not a bug.
Usually, you have these choices:

the "clean" way: use 32-bit Python if you have to use 32-bit modules or extensions
the other "clean" way: only use 64-bit installers when using 64-bit Python (see below)
what the answer above suggests: copy HKLM\SOFTWARE\Python to HKLM\SOFTWARE\wow6432node\Python, but this will cause problems with binary distributions, as 64-bit Python can't load 32-bit compiled modules (do NOT do this!)
install pure Python modules with setuptools instead of the distutils installer (easy_install or pip)

For setuptools itself, for example, you can't use a 32-bit installer for 64-bit Python as it includes binary files. But there's a 64-bit installer at http://www.lfd.uci.edu/~gohlke/pythonlibs/ (has many installers for other modules too). As a rule of thumb: install pure-python modules using distribute and pip (see here), and find a binary package matching your installed Python architecture for everything else.


Input:
How can I reference requirements.txt for the install_requires kwarg in setuptools' setup.py file? I have a requirements.txt file that I'm using with Travis-CI.  It seems silly to duplicate the requirements in both requirements.txt and setup.py, so I was hoping to pass a file handle to the install_requires kwarg in setuptools.setup.
Is this possible?
If so, how should I go about doing it?
For good measure, here is my requirements.txt file:
guessit>=0.5.2
tvdb_api>=1.8.2
hachoir-metadata>=1.3.3
hachoir-core>=1.3.3
hachoir-parser>=1.3.4

Output:
A requirement file can contain comments (#) and can include some other files (--requirement or -r).
Thus, if you really want to parse a requirement.txt you should use the pip parser:
from pip.req import parse_requirements

# parse_requirements() returns generator of pip.req.InstallRequirement objects
install_reqs = parse_requirements(<requirements_path>)

# reqs is a list of requirement
# e.g. ['django==1.5.1', 'mezzanine==1.4.6']
reqs = [str(ir.req) for ir in install_reqs]

setup(
    ...
    install_requires=reqs
)

NOTE: my answer is old now. Pip does not have a public API so this no longer works (e.g. parse_requirements now requires an instance of pip.download.PipSession). You can do the opposite: list the dependances in setup.py and have a single character -- a dot . -- in requirements.txt.


Input:
Catching an exception while using a Python 'with' statement To my shame, I can't figure out how to handle exception for python 'with' statement. If I have a code:
with open("a.txt") as f:
    print f.readlines()

I really want to handle 'file not found exception' in order to do somehing. But I can't write
with open("a.txt") as f:
    print f.readlines()
except:
    print 'oops'

and can't write
with open("a.txt") as f:
    print f.readlines()
else:
    print 'oops'

enclosing 'with' in a try/except statement doesn't work else: exception is not raised. What can I do in order to process failure inside 'with' statement in a Pythonic way?

Output:
from __future__ import with_statement

try:
    with open( "a.txt" ) as f :
        print f.readlines()
except EnvironmentError: # parent of IOError, OSError *and* WindowsError where available
    print 'oops'

If you want different handling for errors from the open call vs the working code you could do:
try:
    f = open('foo.txt')
except IOError:
    print('error')
else:
    with f:
        print f.readlines()


Input:
Building a minimal plugin architecture in Python I have an application, written in Python, which is used by a fairly technical audience (scientists).
I'm looking for a good way to make the application extensible by the users, i.e. a scripting/plugin architecture.
I am looking for something extremely lightweight. Most scripts, or plugins, are not going to be developed and distributed by a third-party and installed, but are going to be something whipped up by a user in a few minutes to automate a repeating task, add support for a file format, etc. So plugins should have the absolute minimum boilerplate code, and require no 'installation' other than copying to a folder (so something like setuptools entry points, or the Zope plugin architecture seems like too much.)
Are there any systems like this already out there, or any projects that implement a similar scheme that I should look at for ideas / inspiration?

Output:
Mine is, basically, a directory called "plugins" which the main app can poll and then use imp.load_module to pick up files, look for a well-known entry point possibly with module-level config params, and go from there. I use file-monitoring stuff for a certain amount of dynamism in which plugins are active, but that's a nice-to-have.
Of course, any requirement that comes along saying "I don't need [big, complicated thing] X; I just want something lightweight" runs the risk of re-implementing X one discovered requirement at a time. But that's not to say you can't have some fun doing it anyway :)


Input:
Django: Redirect to previous page after login I'm trying to build a simple website with login functionality very similar to the one here on SO.
The user should be able to browse the site as an anonymous user and there will be a login link on every page. When clicking on the login link the user will be taken to the login form. After a successful login the user should be taken back to the page from where he clicked the login link in the first place.
I'm guessing that I have to somehow pass the url of the current page to the view that handles the login form but I can't really get it to work.
EDIT:
I figured it out. I linked to the login form by passing the current page as a GET parameter and then used 'next' to redirect to that page. Thanks!
EDIT 2:
My explanation did not seem to be clear so as requested here is my code:
Lets say we are on a page foo.html and we are not logged in. Now we would like to have a link on foo.html that links to login.html. There we can login and are then redirected back to foo.html.
The link on foo.html looks like this:
      Login

Now I wrote a custom login view that looks somewhat like this:
def login_view(request):
   redirect_to = request.REQUEST.get('next', '')
   if request.method=='POST':
      #create login form...
      if valid login credentials have been entered:
         return HttpResponseRedirect(redirect_to)
   #...
   return render_to_response('login.html', locals())

And the important line in login.html:
<form method="post" action="./?next={{ redirect_to }}">

So yeah thats pretty much it, hope that makes it clear.

Output:
You do not need to make an extra view for this, the functionality is already built in.
First each page with a login link needs to know the current path, and the easiest way is to add the request context preprosessor to settings.py (the 4 first are default), then the request object will be available in each request:
settings.py:
TEMPLATE_CONTEXT_PROCESSORS = (
    "django.core.context_processors.auth",
    "django.core.context_processors.debug",
    "django.core.context_processors.i18n",
    "django.core.context_processors.media",
    "django.core.context_processors.request",
)

Then add in the template you want the Login link:
base.html:
<a href="{% url django.contrib.auth.views.login %}?next={{request.path}}">Login</a>

This will add a GET argument to the login page that points back to the current page.
The login template can then be as simple as this:
registration/login.html:
{% block content %}
<form method="post" action="">
  {{form.as_p}}
<input type="submit" value="Login">
</form>
{% endblock %}


Input:
Is False == 0 and True == 1 in Python an implementation detail or is it guaranteed by the language? Is it guaranteed that False == 0 and True == 1, in Python?  For instance, is it in any way guaranteed that the following code will always produce the same results, whatever the version of Python (both existing and, likely, future ones)?
0 == False  # True
1 == True   # True
['zero', 'one'][False]  # is 'zero'

Any reference to the official documentation would be much appreciated!
Edit: As noted in many answers, bool inherits from int.  The question can therefore be recast as: "Does the documentation officially say that programmers can rely on booleans inheriting from integers, with the values 0 and 1?".  This question is relevant for writing robust code that won't fail because of implementation details!

Output:
In Python 2.x this is not guaranteed as it is possible for True and False to be reassigned.  However, even if this happens, boolean True and boolean False are still properly returned for comparisons.
In Python 3.x True and False are keywords and will always be equal to 1 and 0.
Under normal circumstances in Python 2, and always in Python 3:
False object is of type bool which is a subclass of int:
object
   |
 int
   |
 bool

It is the only reason why in your example, ['zero', 'one'][False] does work. It would not work with an object which is not a subclass of integer, because list indexing only works with integers, or objects that define a __index__ method (thanks mark-dickinson).
Edit:
It is true of the current python version, and of that of Python 3. The docs for python 2.6 and the docs for Python 3 both say:

There are two types of integers: [...] Integers (int) [...] Booleans (bool)

and in the boolean subsection:

Booleans: These represent the truth values False and True [...] Boolean values behave like the values 0 and 1, respectively, in almost all contexts, the exception being that when converted to a string, the strings "False" or "True" are returned, respectively.

So booleans are explicitly considered as integers in Python 2.6 and 3.
So you're safe until Python 4 comes along. ;-)


Input:
Decode HTML entities in Python string? I'm parsing some HTML with Beautiful Soup 3, but it contains HTML entities which Beautiful Soup 3 doesn't automatically decode for me:
>>> from BeautifulSoup import BeautifulSoup

>>> soup = BeautifulSoup("<p>£682m</p>")
>>> text = soup.find("p").string

>>> print text
£682m

How can I decode the HTML entities in text to get "Â£682m" instead of "£682m".

Output:
Python 3.4+
HTMLParser.unescape is deprecated, and was supposed to be removed in 3.5, although it was left in by mistake. It will be removed from the language soon. Instead, use html.unescape():
import html
print(html.unescape('£682m'))

see https://docs.python.org/3/library/html.html#html.unescape

Python 2.6-3.3
You can use the HTML parser from the standard library:
>>> try:
...     # Python 2.6-2.7
...     from HTMLParser import HTMLParser
... except ImportError:
...     # Python 3
...     from html.parser import HTMLParser
...
>>> h = HTMLParser()
>>> print(h.unescape('£682m'))
Â£682m

See http://docs.python.org/2/library/htmlparser.html
You can also use the six compatibility library to simplify the import:
>>> from six.moves.html_parser import HTMLParser
>>> h = HTMLParser()
>>> print(h.unescape('£682m'))
Â£682m