Unittesting with Pyspark: unclosed socket warnings
These warnings will go away if you add the instruction to ignore them in the setUp method of the test:
import unittestimport warningsclass MyTest(unittest.TestCase): def test(self): val = 1 self.assertEqual(insert_and_collect(val), val) print('four') def setUp(self): warnings.filterwarnings("ignore", category=ResourceWarning) warnings.filterwarnings("ignore", category=DeprecationWarning)Now running with python3 -m unittest pyspark_unittesting.py outputs:
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.propertiesSetting default log level to "WARN".To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).one1 twothreefour.----------------------------------------------------------------------Ran 1 test in 11.124sOK