Short answer: The simplest solution is to write the complete list to file using pickle.dump()
. There’s no need to write all objects one by one in a loop. Pickle is designed to do this for you.
Example code and alternative solutions:
Below is a fully working example. Some notes:
- I’ve updated your
__init__
function a bit to make the initialization code a lot easier and shorter. - I’ve also added a
__repr__
function. This could be used to print the record details to screen, which you also asked. (Note that you could also implement a__str__
function, but I chose to implement__repr__
for this example). - This code example uses standard Python coding styles (PEP-8).
- This code uses a context manager to open the file. This is safer and avoid the need to manually close the file.
If you really want to write the objects manually, for whatever reason, there are a few alternatives to do that safely. I’ll explain them after this code example:
import pickle
class CarRecord:
def __init__(self, vehicle_id, registration, registration_date, engine_size, purchase_price):
self.vehicle_id = vehicle_id
self.registration = registration
self.registration_date = registration_date
self.engine_size = engine_size
self.purchase_price = purchase_price
def __repr__(self):
return "CarRecord(%r, %r, %r, %r, %r)" % (self.vehicle_id, self.registration,
self.registration_date, self.engine_size,
self.purchase_price)
def main():
cars = [
CarRecord("CD333", "17888", "18/2/2017", 2500, 22000.00),
CarRecord("AB123", "16988", "19/2/2017", 2500, 20000.00),
]
# Write cars to file.
with open('Cars.TXT', 'wb') as car_file:
pickle.dump(cars, car_file)
# Read cars from file.
with open('Cars.TXT', 'rb') as car_file:
cars = pickle.load(car_file)
# Print cars.
for car in cars:
print(car)
if __name__ == '__main__':
main()
Output:
CarRecord('CD333', '17888', '18/2/2017', 2500, 22000.0)
CarRecord('AB123', '16988', '19/2/2017', 2500, 20000.0)
Instead of dumping the list at once, you could also do it in a loop. The following code snippets are alternative implementations to «Write cars to file» and «Read cars from file».
Alternative 1: write number of objects to file
At the start of the file, write the number of cars. This can be used to read the same amount of cars from the file.
# Write cars to file.
with open('Cars.TXT', 'wb') as car_file:
pickle.dump(len(cars), car_file)
for car in cars:
pickle.dump(car, car_file)
# Read cars from file.
with open('Cars.TXT', 'rb') as car_file:
num_cars = pickle.load(car_file)
cars = [pickle.load(car_file) for _ in range(num_cars)]
Alternative 2: use an «end» marker
At the end of the file, write some recognizable value, for example None
. When reading this object can be used to detect the end of file.
# Write cars to file.
with open('Cars.TXT', 'wb') as car_file:
for car in cars:
pickle.dump(car, car_file)
pickle.dump(None, car_file)
# Read cars from file.
with open('Cars.TXT', 'rb') as car_file:
cars = []
while True:
car = pickle.load(car_file)
if car is None:
break
cars.append(car)
Уведомления
- Начало
- » Python для новичков
- » Ran out of input
#1 Март 30, 2016 13:59:14
Ran out of input
def top_scores(score): """Forms a top scores list""" name = input("Enter your name") f = open("pickles11.dat", "wb+") top_scores = pickle.load(f) top_scores[score] = name pickle.dump(top_scores, f) f.close() keys = top_scores.keys() keys = list(keys) keys.sort(reverse=True) print("nTop scores:") for i in keys: print(top_scores[i], i)
При выполнении возникает ошибка:
top_scores = pickle.load(f)
EOFError: Ran out of input
Подскажите, пожалуйста, где здесь проблема.
Офлайн
- Пожаловаться
#2 Март 30, 2016 14:56:55
Ran out of input
Stranger
f = open("pickles11.dat", "wb+")
wb+ стирает содержимое файла.
Офлайн
- Пожаловаться
#3 Март 30, 2016 15:02:59
Ran out of input
Сначала получаем значение, потом его изменяем и вновь записываем, wb+ это позволяет. Я не прав?
P.S. Да и в принципе не важно, что оно делает, просто подскажите, как сделать, чтобы всё заработало
Отредактировано Stranger (Март 30, 2016 15:06:38)
Прикреплённый файлы:
Снимок экрана от 2016-03-30 15-00-47.png (212,3 KБ)
Офлайн
- Пожаловаться
#5 Март 30, 2016 15:25:37
Ran out of input
Ну так в файл перезаписывается уже полученная и измененная ценность, или я сильно ошибаюсь?
Офлайн
- Пожаловаться
#6 Март 30, 2016 15:31:15
Ran out of input
Stranger
Откройте файл с wb+ и прочтитайте его содержимое.
Согласитесь, проще сделать это, чем гадать?
Офлайн
- Пожаловаться
#7 Март 30, 2016 15:47:54
Ran out of input
def top_scores(score): """Forms a top scores list""" name = input("Enter your name") f = open("pickles11.dat", "rb") top_scores = pickle.load(f) top_scores[score] = name f.close() f = open("pickles11.dat", "wb") pickle.dump(top_scores, f) f.close() keys = top_scores.keys() keys = list(keys) keys.sort(reverse=True) print("nTop scores:") for i in keys: print(top_scores[i], i)
Да, вы правы, спасибо за помощь. Переписал, всё работает.
Офлайн
- Пожаловаться
У меня в коде изначально всё работает. При повторном запуске, когда уже создан файл, в котором хранятся данные, вылезает ошибка Run out of input.
user_list = []
password_list = []
lesson_assessments = dict()
try:
opn_f = open('datafile.data', 'rb')
except IOError:
None
else:
user_list = pickle.load(opn_f)
password_list = pickle.load(opn_f)
lesson_assessments =pickle.load(opn_f)
ошибка в строке с password_list = pickle.load(opn_f)
Issue Type: Bug
Error 2020-07-22 20:17:13: stderr jediProxy Error (stderr) Traceback (most recent call last):
File "c:Usersaaa.vscodeextensionsms-python.python-2020.7.94776pythonFileslibpythonjediinferencecompiledsubprocess__init__.py", line 261, in _send
is_exception, traceback, result = pickle_load(self._get_process().stdout)
File "c:Usersaaa.vscodeextensionsms-python.python-2020.7.94776pythonFileslibpythonjedi_compatibility.py", line 396, in pickle_load
return pickle.load(file, encoding='bytes')
EOFError: Ran out of input
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "c:Usersaaa.vscodeextensionsms-python.python-2020.7.94776pythonFileslibpythonjediapienvironment.py", line 75, in _get_subprocess
info = self._subprocess._send(None, _get_info)
File "c:Usersaaa.vscodeextensionsms-python.python-2020.7.94776pythonFileslibpythonjediinferencecompiledsubprocess__init__.py", line 273, in _send
stderr,
jedi.api.exceptions.InternalError: The subprocess D:Program-FilesAnaconda3envspy36-ideapython.exe has crashed (EOFError('Ran out of input',), stderr=).
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "c:Usersaaa.vscodeextensionsms-python.python-2020.7.94776pythonFilescompletion.py", line 661, in watch
response = self._process_request(rq)
File "c:Usersaaa.vscodeextensionsms-python.python-2020.7.94776pythonFilescompletion.py", line 589, in _process_request
all_scopes=True,
File "c:Usersaaa.vscodeextensionsms-python.python-2020.7.94776pythonFileslibpythonjediapi__init__.py", line 885, in names
return Script(source, path=path, encoding=encoding).get_names(
File "c:Usersaaa.vscodeextensionsms-python.python-2020.7.94776pythonFileslibpythonjediapi__init__.py", line 185, in __init__
project, environment=environment, script_path=self.path
File "c:Usersaaa.vscodeextensionsms-python.python-2020.7.94776pythonFileslibpythonjediinference__init__.py", line 90, in __init__
self.compiled_subprocess = environment.get_inference_state_subprocess(self)
File "c:Usersaaa.vscodeextensionsms-python.python-2020.7.94776pythonFileslibpythonjediapienvironment.py", line 114, in get_inference_state_subprocess
return InferenceStateSubprocess(inference_state, self._get_subprocess())
File "c:Usersaaa.vscodeextensionsms-python.python-2020.7.94776pythonFileslibpythonjediapienvironment.py", line 80, in _get_subprocess
exc))
jedi.api.environment.InvalidPythonEnvironment: Could not get version information for 'D:\Program-Files\Anaconda3\envs\py36-idea\python.exe': InternalError("The subprocess D:\Program-Files\Anaconda3\envs\py36-idea\python.exe has crashed (EOFError('Ran out of input',), stderr=).",)
Extension version: 2020.7.94776
VS Code version: Code 1.47.2 (17299e413d5590b14ab0340ea477cdd86ff13daf, 2020-07-15T18:22:06.216Z)
OS version: Windows_NT x64 10.0.18363
System Info
Item | Value |
---|---|
CPUs | Intel(R) Core(TM) i7-6500U CPU @ 2.50GHz (4 x 2592) |
GPU Status | 2d_canvas: enabled flash_3d: enabled flash_stage3d: enabled flash_stage3d_baseline: enabled gpu_compositing: enabled multiple_raster_threads: enabled_on oop_rasterization: disabled_off protected_video_decode: unavailable_off rasterization: enabled skia_renderer: disabled_off_ok video_decode: enabled viz_display_compositor: enabled_on viz_hit_test_surface_layer: disabled_off_ok webgl: enabled webgl2: enabled |
Load (avg) | undefined |
Memory (System) | 15.89GB (7.72GB free) |
Process Argv | E:downloadschrome |
Screen Reader | no |
VM | 0% |
Why did multiprocessing giving me EOFError : Ran out of Input Error?
videoPlayerThreading
is my own made library to basically making 2 class with each using threading to get and show frames. objDect
is also my own library to basically return frame after object detection. I got EOFError : Ran out of Input
error and from the traceback I think it is caused by the multiprocessing itself hence I don’t post my library because it is so long. Can anyone help me with what is wrong with it? Thank you
from multiprocessing import Process import sys import videoPlayerThreading as vpt from objDect import objDect as od def main(videoSource): obd = od( videoSources = videoSource ) getFrame = vpt.getFrames(videoSource).start() showFrame = vpt.showFrames(videoSource).start() while True: frame = getFrame.frame frame=Process(target=obd.predictYolo, args=(frame,)).start() showFrame.frame = frame if getFrame.doVideo == False or showFrame.doVideo == False: getFrame.stop() showFrame.stop() sys.exit() if __name__=="__main__": main(0)
Edit :
Here is the show frames and get frames class it basically only get and show frame using threading.
class getFrames(): def __init__(self, videoSource:Union[int,str]=0): self.stream = self.videoInit(videoSource) self.hasFrame, self.frame = self.stream.read() self.doVideo = True def videoInit(self, videoSource:Union[int,str]): try: cap = cv2.VideoCapture(videoSource) except Exception as e: raise Exception(f"Video source error: {e}") return cap def start(self): Thread(target=self.getFrames, args=()).start() return self def getFrames(self): while self.doVideo: if not self.hasFrame: self.stop() else: (self.hasFrame, self.frame) = self.stream.read() def stop(self): self.doVideo = False self.stream.release() class showFrames(): def __init__(self, frame:cv2=None): self.frame = frame self.doVideo = True def start(self): Thread(target=self.showFrame, args=()).start() return self def showFrame(self): while self.doVideo: cv2.imshow("Video", self.frame) if cv2.waitKey(1) == ord("q"): self.doVideo = False def stop(self): self.doVideo = False
Advertisement
Answer
The best I can understand your program logic you need something like the following. Generator function read_frames
(which may or may not need correction), reads the frames one by one yielding each frame. The main process creates a multiprocessing pool and passes each input frame to the multiprocessing pool to be processed by obd.predictYolo
and sets vpt.frame
with the returned frame. This continues until either there are no more frames to process or showFrame.doVideo
is False
. In short, I have done away with your getFrames
class, which is useless here.
I do not have OpenCV installed and do not really know the package nor do I have your video file, so consider this a starting point for your further investigation.
from multiprocessing.pool import Pool import sys import videoPlayerThreading as vpt from objDect import objDect as od def read_frames(videoSource:Union[int,str]=0): try: stream = cv2.VideoCapture(videoSource) except Exception as e: raise Exception(f"Video source error: {e}") while True: hasFrame, frame = stream.read() if not hasFrame: break yield frame def main(videoSource): obd = od( videoSources = videoSource ) showFrame = vpt.showFrames(videoSource).start() with Pool() as pool: for frame in pool.imap(obd.predictYolo, read_frames(videoSource)): showFrame.frame = frame if showFrame.doVideo is False: showFrame.stop() break if __name__=="__main__": main(0)