Resolve `TypeError can't pickle thread.lock objects`

This article is for Python 2.7 only. May be incorrect for Python 3.x.

1. super() should not be referenced.

Bad practice:

class foo(object):
	pass

class bar(foo):
	def __init__(self, a):
		self.p = super(bar, self)
		self.p.__init__(a)

This will throw TypeError: can't pickle thread.lock objects will be raised if you try to dump an object of class bar.

Good practice:

class foo(object):
	def __init__(self, a):
		pass

class bar(foo):
	def __init__(self, a):
		self.p = super(bar, self).__init__(a)

2. Caused by Logging.

If you are using build-in logging module together with th build-in pickle module, this exeption may casued by the conflict between them.

Bad practice:

import logging
class cannot_pickle(object):
	def __init__(self):
		self.logger = logging.getLogger('test')
		
	def log(self, message):
		self.logger.info(message)

logger object cannot be dumped by Pickle in Python2.7. The logging module implements a thread-safe logging mechanism with thread.lock in it. To resolve this issue, you may use the getLogger() method instead of an explict reference to a logger object.

Good practice:

import logging
class can_pickle(object):
	def __init__(self):
		pass

	@property
	def logger(self):
		return logging.getLogger('test')
		
	def log(self, message):
		self.logger.info(message)

3. If you are using sklearn.

Sometimes your may need to override an existing sklearn.estimator. Remember that if you are adding any new parameter to your wrapper class, please set them as object variables in __init__().

See the following example of overriding sklearn.preprocessing.Imputer.

Bad practice:

from sklearn.preprocessing import Imputer

class override_imputer(Imputer):
	def __init__(self, 
			foo=None, bar=0, # added paramters
         	missing_values="NaN", strategy="mean", axis=0, verbose=0, copy=True, # Imputer()'s parameters
	        	):
		super(override_imputer, self).__init__(missing_values=missing_values, strategy=strategy,
                 axis=axis, verbose=verbose, copy=copy)

Good practice:

from sklearn.preprocessing import Imputer

class override_imputer(Imputer):
	def __init__(self, 
			foo=None, bar=0, # added paramters
         	missing_values="NaN", strategy="mean", axis=0, verbose=0, copy=True, # Imputer()'s parameters
	        	):
		super(override_imputer, self).__init__(missing_values=missing_values, strategy=strategy,
                 axis=axis, verbose=verbose, copy=copy) 
      	'''
      	Remember to do the following:
      	'''
      	self.foo = foo
      	self.bar = bar

---

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.