nzbToMedia/changelog.txt

Change_LOG / History

V12.0.7

Refactor utils
Fix git subprocess
Fix cleanup script output
Add extra logging for fork detection
Additional code clean up

V12.0.6

Hotfix for Manual Torrent run results.

V12.0.5

Proper fix for source cleaner

V12.0.4

Hotfix missed commit for source cleaner

V12.0.3

Hotfix cleaning for source installs

V12.0.2

Fix missed ProcessResult

V12.0.1

Added Python 3 support
Updated all dependencies
Major code refactoring
Various bug fixes
Hotfix NZBGet not working without comment

V12.0.0

NOTE:
- This release contains major backwards-incompatible changes to the internal API
- Windows users will need to manually install pywin32

Add Python 3 support
Add cleanup script for post-update cleanup
Update all dependencies
Move vendored packages in `core` to `libs`
Move common libs to `libs/common`
Move custom libs to `libs/custom`
Move Python 2 libs to `libs/py2`
Move Windows libs to `libs/windows`
Fix PEP8
Add feature to make libs importable
Add feature to auto-update libs
Add path parent option to module path and default to using local path
Update invisible.cmd to return errorlevel
Update invisible.vbs to return exit code of 7zip
Update extractor.py for correct return code
Added debugging to extractor
Add option for windows extraction debugging
Remove surplus debug
Fix handling of None Password file
Fix invisible windows extraction
Fix execution of extraction
Start vbs directly from extractor
Delete invisible.cmd
Use args instead of Wscript.Arguments
Fix postprocessing of failed / bad downloads (#1091)
Fix release is None
Fix UnRAR failing

V11.8.1 12/29/2018

Fix cleanup for nzbToMedia installed as a git submodule

V11.8.0 12/28/2018

Add version information
Add bumpversion support
Fix automatic cleanup script

V11.7 12/25/2018

Merry Christmas and Happy Holidays!

Add cleanup script to clean up bytecode
Add automatic cleanup on update

NOTE: Cleanup will force-run every time during a transitional period to minimize issues with upcoming refactoring

V11.06 11/03/2018

updates to incorporate importMode for NzbDrone/Sonarr and Radarr.
Correct typo(s) for "Lidarr" category.
only pass id to CP if release id found.
fix issue with no release id and no imdbid.
Fixed NZBGet save of Lidarr config.
improve logging for imdb id lookup.
fix minor description error.
add better logging of movie name when added to CP.
attempt to clean up Liddar api commands.
update to use Mylar api.
set Torrent move-sym option to force SickRage process_method.
add rmDir import for HeadPhones processing.
change sickrage and sickchill names and modify api process to work with multiple sick* forks.
add NZBGet WebUI set of delete failed for HP.
fix qbittorrent to delete permanently (remove files on delete).

V11.05 27/06/2018

Add qBittorrent support.
Add SickGear support.
Add SiCKRAGE api support.
Fix for single file download.
Diable media check for failed HeadPhones downloads.
Added Lidarr flow. Still awaiting confirmation of api interface commands and return.

V11.04 30/12/2017

do not embed .sub.
add proper check of sub streams #1150 and filter out commentary.
traverse audiostreams in reverse.
add catch for OMDB api errors.
convert all listdir functions to unicode.
perform extraction, corruption checks, and transcoding when no server.
fix list indices errors when no fork set.
fix CP server responding test. Add trailing /.
use basestring to match unicode path in transcoder.
attempt autofork even if no username set.
allow long paths in Cleandir.
add Radarr handling.
minor fix for transcoder.
fix non-iterable type.
fix logging error.
DownloadedMovieScan updated to DownloadedMoviesScan.
add check to exception rename to not over-write exisiting.
don't try and process when no api/user.
Added omdbapikey functionality
force sonarr processing to "move".
already extracted archive not skipped.
fix text for keep_archive.
try to avoid spaces in outputdir.
change subtitle logging level.
Increase shutil copy buffer length from 4KB to 512KB.
improve user script media extension handling
add par2 rename/repair (linux only).

V11.03 15/01/2017

Add -o to output path for 7zip.
Try album directory then parent directory for HeadPhones variants.
Prevent duplication of audio tracks in Transcoder.
Update uTorrent Client interface.
Updated to use force_next for SickRage to prevent postprocessing in queue.

V11.02 30/11/2016

Added default "MKV-SD"
Added VideoResolution in nzbGet.
Fix Headphones direcotry parsing.
Remove proc_type when failed.
Added option "no_extract_failed"
Updated beautifulsoup 4 module.
Check for existence of codec_type key when counting streams.
Added default fallback for sabnzbd port = 8080.

V11.01 30/10/2016

Updated external modules and changed config to dict.
Started making code python 3 compatible.
Fixed auto-fork detection for new Sick* branches.
Fixed invalid indexing scope for TorrentToMedia.
Add Medusa fork and new param "ignore_subs".
Added check for language tag size, convert 3 letter language codes.
Fixed guessit call to allow guessit to work of full file path.
Add the ability to set octal permissions on the processed files prior to handing it off to Sickrage/Couchpotato.
Catch errors if not audio codec name.
Allow manual scans to continue.
Revert to 7zip if others missing.
Fixed int conversion base 8 from string or int.
Added more logging to server tests.
Added MKV-SD Profile.
Check for preferred codec even if not preferred language.
Don't convert VobSub to mov_text.

V10.15 29/05/2016

Don't copy archives when set to extract.
Specifically check for failed download handing regardless of fork.
sort Media file results by pathlength.
Synchronize changed SickRage directory param.
Don't remove release group information from base folder.
Don't add imdb id to file name when move-sym in use.
Fix string and integer concat error.

V10.14 13/03/2016

Add option move-sym to create symlink to renamed files.
Transmission comment fix.
Prevent int errors in chmod.
Fix urllib warnings.
Create unique directory in output incase of rename error in sick/couch.
Add -strict -2 to dts codec.
Added support to handle archives in SickRage.
Report Downloader failures to SickRage.
Continue on encoding detection failure.
Strip trailing and leading whitespaces from `mount_points`.
Also check sabnzbd history for nzoid.
Add generic run mode (manually enter parameters for execution).

V10.13 11/12/2015

Always add -strict -2 to aac codec.
Add "delete_on" for SickRage.
Add https handling for SABnzbd.
Added the ability to chmod Torrent diretory before processing.
Add option to not resume failed torrent.
Add Option to not resume successful torrent.
Add procees name to final SABnzbd message.
Fix SSL warnings forcomic processing.
Add .ts to mediaExtensions.
Don't update plex on failed.
Add option to preserve archive files after extraction.
Force_Clean doesn't over-ride delete_failed.
Added support for SickRageTV and SickRage branches.

V10.12 21/09/2015

Updated Requests Module to Latest Version. Works with Python 2.7.10
Add .img files to transcoder extraction routines.

V10.11 28/05/2015

Use socket to verify if running on Linux. Prevents issues with stale pid.
Add timeouts and improve single instance handling.
Prevent Scale Up.
Improve regex for rename script.
Improve safe rename functionality.
Ignore .bts extensions.
Don't process output when no transcoding needed.
Ignore Thumbs.db on manual run.
Rename nzbtomedia to core. To prevent erros on non-case sensitive file systems.
Mark as bad if no media files found.
Increase server responding timeout.
Don't use last modified entry for CP renamer when no imdb id found.
Add plex library update.

V10.10 29/01/2015

Fix error when extracting on windows. (added import of subprocess)
Fix subtitles download and emdedding.

V10.9 19/01/2015

Prevent Errors when trying next release from CouchPotato (CouchPotato failed handling)
Prevent check for status change when using Manage scan (CouchPotato)
Better Tooltip for "host" in NZBGet settings.
Continue if failed to connect to Torrent Client.
Fixed resolution settings in Transcoder.
Make Windows Linking and extraction invisible.

V10.8 15/12/2014

Impacts All
Removed "stand alone" scripts DeleteSamples and ResetDateTimes. These are now in https://github.com/clinton-hall/GetScripts
Removed chp.exe and replaced with vb script.
Improved Sonarr(NZBDrone) CDH support.
Use folder Permissions to set permissions for sub directories and files following extract.
Added support fro new SickRage Login.

Impacts NZBs
Get NZOID from SABnzbd for better release matching.

Impacts Torrents
Now gets Label from Deluge.
Changed SSL version for updated Deluge (0.3.11+)

Impacts Transcoding
Fixed reported bugs.
Fix Audio mapping.
Fix Subtitle mapping from external files.
Fixed scaling errors.

V10.7 06/10/2014

Impacts All
Add Transcoding of iso/images and VIDEO_TS structures.
Improved multiple session handling.
Improve NZBDrone handling (including Torrent Branch).
Multiple bug-fixes.

Impacts NZBs
Add custom "group" replacements to allow better subtitle searching.

Impacts Torrents
Add Vuze Torrent Client support.

V10.6 26/08/2014

Impacts All
Bug Fixes.

Impacts NZBs
Added FailureLink style feedback to dognzb for failed and corrupt downloads.

V10.5 05/08/2014

Impacts All
Bug Fixes for Transcoder.
Support for lib-av as well as ffmpeg.
Fixed SickBeard aut-fork detection.

V10.4 30/07/2014

Impacts All
Supress printed messages from extractor.
Allow no sub languages to be specified.
Ignore hdmv_pgs_subtitle codecs in transcoder.
Fix remote directory use with HeadPhones.
Only use nice and ionice when available.

Impacts NZBs
Cleaner exit logging for SABnzbd.

Impacts Torrents
Improved manual run handling.

V10.3 15/07/2014

Impacts All
Fix auto-fork to identify default fork.

V10.2 15/07/2014

Impacts All
Bug Fixes.
If extracting files and extraction not successful, return Failure and Don't delete archives.

V10.1 11/07/2014

Impacts All
Improved Transcoder
Minor Bug Fixes
Now accepts Number of Audio Channels for Transcoder options.
Userscript can perform video corruption check first.
Improved extraction. Extract all subdirs and multiple "unique" archives in a directory.
Check if already running and wait for complete before continuing.

Impacts NZBs
Allow UserScript for NZBs

Impacts Torrents
Do Extraction Before Flatten


V10.0 03/07/2014

Impacts All
Changed to python2 (some systems now come with python = python3 as default).
Major changes to Transcoder. Only copy streams where possible.
Pre-defined Transcode options for some devices.
Added log_env option to capture environment variables.
Improved remote directory handling.
Various fixes.

V9.3 09/06/2014

Impacts Torrents
Allow Headphones to remove torrents and data after processing.
Delete torrent if uselink = move
Added forceClean for outputDir. Works in file permissions prevent CP/SB from moving files.
Ignore .x264 from archive "part" checks.
Changed handling of TPB/Pistachitos SB forks. Default is to link/extract here. Disabled by Torrent_NoLink = 1.
Changed handling for HeadPhones Now that HeadPhones allows process directory to be defined.
Restructured Flow and streamlines process

Impacts NZBs
Fix setting of Mylar config from NZBGet.
Created sheel scripts to nzbTo{App}. All now call the common nzbToMedia.py

Impacts All
Changes to Couchpotato API for [nosql] added. Keeps aligned with current CouchPotato develop branch.
Add Auto Detection of SickBeard Fork. Thanks @echel0n
Added config class, re-coded migratecfg, misc bugfixes and code cleanup.
Added dynamic timeout based on directory size.
Added process_Method for SickBeard.
Changed configuration migrate process.
Major structure and process re-format.
Improved Manual Call Handling
Now prints github version into log when available.
Changed log location and format.
Added autoUpdate option via git.
All calls now use requests, not urllib.
All details now saved into Database. Can be used for more features later ;)
Improved status checking to ensure we only cleanup when successfully processed.

Huge Thanks @echel0n

V9.2 05/03/2014

Impacts All
Change default "wait_for" to 5 mins. CouchPotato can take more than 2 minutes to return on renamer.scan request.
Added SickBeard "wait_for" to bw customizable to prevent unwanted timeouts.
Fixed ascii conversion of directory name.
Added list of common sample ids and a way to set deletion of All media files less than the sample file size limit.
Added urlquote to dirName for CouchPotato (allows special characters in directory name)

Impacts NZBs
Fix Error with manual run of nzbToMedia
Make sure SickBeard receives the individula download dir.
Added option to set SickBeard extraction as either Downlaoder or Destination (SickBeard).
Fixed Health Check handling for NZBGet.

Impacts Torrents
Added option to run userscript once only (on directory).
Added Option to not flatten specific categories.
Added rtorrent integration.
Fixes for HeadPhones use (no flatten), no move/sym, and fix move back to original.

V9.1 24/01/2014

Impacts All
Don't wait to verify status change in CouchPotato when no initial status (manual run)
Now use "wait_for" timing as socket timeout on the renamer.scan. It appears to now be delayed in confirming success.

V9.0 19/01/2014

Impacts NZBs
SABnzbd 0.7.17+ now uses 8 arguments, not 7. These scripts now support the extra argument.

Impacts Torrents
Always pause before processing.
Moved delete to end of routine, only when succesful process occurs.
Don't flatten hp category (in case multi cd album)
Added UserScript to be called for un-categorized downloads and other defined categories.
Added Torrent Hash to Deluge to assist with movie ID.
Added passwords option to attempt extraction od passworded archives.

Impacts All
Added default socket timeout to prevent script hanging when the destination servers don't respond to http requests.
Made processing Category Centric as an option for people running multiple versions of SickBeard and CouchPotato etc.
Added TPB version of SickBeard processing. This now uses a fork pass-in instead of failed_fork.
Added new option to convert files, directories, and parameters to ASCII. To be used if you regularly download "foreign" titles and have problems with CP/SB.
Now only parse results from CouchPotato 50 at a time to prevent error with large wanted list.

V8.5 05/10/2013

Impacts Torrents
Added Transmission RPC client.
Now pauses and resumes or removes from transmission.
Added debugging of input arguments from torrent clients.

Impacts NZBs
Removed obsolete NZBget (pre V11) code.

Impacts All.
Fixed HeadPhones processing.
Fixed movie parsing in CPS api.

V8.4 14/09/2013

Impacts Torrents
Don't include 720p or 1080p as parts for extracting.
Extracts all sub-folders.
Added option to Move files.
Fix for single file torrents linked to subfolder of same name.

Impacts All
Added option for SickBeard delay (for forks that use 1 minute check.
Updated to new api call in CouchPotato (movie.searcher.try_next)


V8.3 11/07/2013

Impacts All
Allow use of experimental AAC codec in transcoder.
Remove username and password when api key is used.
Add .m4v as media
Added ResetDateTime.py
Manual Opion for Mylar script.
Fixes for Gamez script.

Impacts NZBs
Added option to remove folder path when CouchPotato on different system to downlaoder.
NZBGet v11.0 stable now current.


V8.2 26/05/2013

Impacts All
Add option to set the "wait_for" period. This is how long the script waits to see if the movie changes status in CouchPotato.
minSampleSize now moved to [extensions] section and availabe for nzbs and torrents.
New option in transcoder to use "niceness" on Linux.
Remove excess logging from transcoder.

Impacts NZBs
Added Flatten of input directory and test for media files (including sample deletion) in autoProcessTV

Impacts Torrents
Fixed Delete_Original option
Fix type which caused crash if not sickbeard or couchpotato.


V8.1 04/05/2013

Impacts All
Improved exception logging for error conditions

Impacts Torrents
Fixed an import error when extracting

Impacts NZBs
Fixed passthrough of inputName from NZBGet to pass the .nzb extension (required for SickBeard's failed fork)


V8.0 28/04/2013

Impacts All
Added download_id pass through for CouchPotato release matching
Uses single directory scanning for CouchPotato renamer
Matches imdb_id, download_id, clientAgent with CPS database

Impacts NZB
Addeed direct configuration support via nzbget webUI (nzbget v11+)
All nzb scripts are now directly callabale in nzbget v11
Settings made in nzbget webUI will be applied to the auotPorcessMedia.cfg when the scripts are run from nzbget.
Fixed TLS support for NZBGet email notifications (for V10 support) 

V7.1 28/03/2013

Impacts Torrents
Added test for chp.exe. If not found, calls 7zip directly
Added test for multi-part archives. Will only extract part1

Impacts NZB
Fixed failed download handling from nzbget (won't delete or move root!!!)
Fixed sendEmail for nzbget to use html with <br> line breaks

V7.0 21/03/2013

Impacts Torrents
Added option to delete torrent and original files after processing (utorrent)

Impacts NZB 
Added nzbget windows script (to be compiled)
Changed nzbget folders to previous X.X, current-stable, testing X.X format
Fix nzbget change directory failure problem
Improved nzbget logging
Add logging to nzbget email notification
Synchronised v10 to latest nzbget testing scripts
Added failed download folder for failed downloads in nzbget
Added option to delete failed in nzbget
Created a single nzbToMedia.py script for all categories (will be the only nzb script compiled for windows)

Impacts All
Added rotating log file handler
Added ffmpeg transcoder
Added CouchPotato status check to provide confirmation of renamer complete 
CouchPotato status check will timeout after 2 minutes in case something goes wrong
Improved logging.
Improved scen exception handling.
Major changes to code layout
Better efficiency
Added support for Mylar, Gamez, and HeadPhones
Moved many of the "support" files to the autoProcess directory so that they aren't visible (looks neater)
Added migration tool to update .cfg file on first run following update.

V6.0 03/03/2013

Impacts Torrents
Bundled 7zip binaries and created extraction functions.
Now pauses uTorrent seeding before calling renamer in SickBeard/CouchPotatoServer
uTorrent Resumes seeding after files (hardlinks) have been renamed

Impacts NZB 
Added local file logging.

Impacts All
Added scene exception handling. Currently for "QoQ"
Improved code layout. 


V5.1 22/02/2013

Improved category search to loop through directory structure.
Added support for deluge and potentially other Torrent clients.
uTorrent now must pass "utorrent" before "%D" "%N" "%L"
added test for date modified (less than 5 mins ago) if root directory and no torrent name.
".cp(ttxxxxxx)" tag preserved in directory name for CPS renaming.
All changes affect Torrent handling. Should not impact NZB handling.

V5.0 20/02/2013

Fixed Extarction and Hard-Linking support in TorrentToMedia
Added new config options for movie file extensions, metadata extensions, compressed file extensions.
Added braid to sync linktastic.
Windows Builds now run without console displaying.
All changes affect Torrent handling. Should not impact NZB handling.

V4.3 17/02/2013

Added Logger in TorrentToMedia.py
Added nzbget V10.0 script.
Delete sample files in nzbget postprocessing
Single Version for all files.

V4.2 12/02/2013

Fixes to TorrentToMedia

V4.1 02/02/2013

Added Torrent Support (µTorrent and Transmission).
Added manual run option for nzbToSickBeard.
Changed nzbGet script to use move not copy and remove.
Merged all .cfg scripts into one (autoProcessMedia.cfg).
Made all scripts execitable (755) on github.
Added category limits for email support in nzbget.
Fixed issue with replacements (of paths) in email messages in nzbget.

V4.0 21/12/2012

Changed name from nzbToCouchPotato to nzbToMedia; Now supports mltiple post-processing from two nzb download clients.
Added email support for nzbget.
Version printing now for each of the nzbTo* scripts.
Added "custom" post-process support in nzbget.
Added post-process script output logging in nzbget.

V3.2 11/12/2012

Added failed handling from NZBGet. Thanks to schumi2004.
Also added support for the "failed download" development branch of SickBeard from https://github.com/Tolstyak/Sick-Beard.git

V3.1 02/12/2012

Added conversion to ensure the status passed to the autoProcessTV and autoProcessMovie is always handled as an integer.

V3.0 30/11/2012

Changed name from sabToCouchPotato to nzbToCouchPotato as this now included NZBGet support.
Packaged the NZBGet postprocess files as well as modified version of nzbToSickBeard (from sabToSickBeard).


V2.2 05/10/2012

Re-wrote the failed downlaod handling to just search for the imdb ttXXXX identifier (as received from the nzb name)
Now issues only two api calls. movie.list and searcher.try_next

Should be more robust with regards changes to CPS and also utilises less resources (i.e. less api call and and less processing).


V2.1 04/10/2012

detected a change in the movie release info format. Fixed the script to work with new format.


V2.0 04/10/2012

Fixed an issue with the failed download handling in that the status id for "snatched" can be different on each installation. now performs a status.list via api to verify the status.

Also including a version print (currently 2.0... yeah original I know) so you know if you are current.

removed the multiple versions. The former _recue version will perform the standard renamer only if "postprocess only verified downloads" (default) is enabled in SABnzbd. Also, the "unix" version works fine in Windows, only the "dos" version gave issue in Linux. In other words, this one version should work for all systems.
For historical reasons, the former download stats apply to the old versions:
sabToCouchPotato-dos - downloaded 143 times
sabToCouchPotato-unix - downloaded 205 times
sabToCouchPotato_recue - downloaded 105 times

Also updated the Windows Build to include the same changes. I have removed the link to the linux build as this didn't work on all systems and it really shouldn't be necessary. Let me know if you need this updated. 


V1.9 18/09/2012

compiled (build) versions of sabToSickBeard and sabToCouchPotato added for both Linux and Windows. links at top of post.


V1.9 16/09/2012

Added a compiled .exe version for windows. Should prevent the "python not recognised" issue and allow this to be used in conjunction with the windows build on systems that do not have python installed.

This is the full (_recue version) if sabnzbd is set to post ptocess only verified jobs, this will not recue and will function as a standard renamer.


V1.9 27/08/2012

Following the latest CPS update on the master branch, this script is not really needed as CPS actually polls the SABnzbd api and does the same as this script (internally).

However, if you have any issues with CPS constantly downloading the same movies, or filling the log with polling SABnzbd for completed movies, or otherwise prefer to use this method, then you can still use this script and make the following changes in CPS:
Settings, renamer, run every (advanced) = set to 1440 (or some longer interval)
Settings, renamer, next On_failed = off
Settings, downloaders, SABnzbd, Delete failed = off.

V1.9 06/08/2012

Also added the integer handling of status in the sabToSickBeard.py script to prevent SickBeard trying to postprocess a failed TV download. Only impacts the _recue version


V1.8 05/08/2012

Modified the _recue version as SABnzbd 0.7.3 now appears to pass the "status" variable as a string not an integer!!! (or i had it wrong on first attempt :~)
This causes the old script to identify completed downloads as failed and recues the next download!

The fix here should work with any conceivable subsequent updates in that I now make the sys.argv[7] an integer before passing it. if the variable already is an integer, this shouldn't cause any issues.

status = int(sys.argv[7])
autoProcessMovie.process(sys.argv[1], sys.argv[2], status)


V1.7 02/08/2012

Added a new version sabToCouchPotato_recue
This works the same as the other versions, but includes support for recuing failed downloads.
This is new, and only tested once (with success  ) at my end.

To get this to run you will need to uncheck the "post-process only verified jobs" option in SABnzbd. Also, to avoid issues with SickBeard postprocessing, I have included a modified postprocessing for SickBeard that just checks for failed status and then exits (the SickBeard Team are currently working on failed download handling and I will hopefully make this script work with that in the future) 

This re-cue works as follows:
Performs an api call to CPS to get a list of all wanted movies (with all data including the releases and status etc)
It finds the nzbname (from SABnzbd) in the json list returned from the api call (movie.list) and identifies the movie id and release id.
It performs an api call to make the release as "ignore" and then performs another api call to refresh the movie.
If another (next best) release that meets your criteria is already available it will send that to SABnzbd, otherwise it will wait until a new release becomes availabe.

I have left the old versions here for now for those who don't want to try this. Also, if you don't uncheck the "post-process only verified jobs" in SABnzbd this code will perform the same as the previous versions.

The next issue to tackle (if this works) is automating the deletion of failed download files in SABnzbd.... but I figured this was a start.


V1.6 22/07/2012

no functionality change, but providing scripts in both unix and dos format to prevent exit(127) errors.
if you are using windows, use the dos format. if you are using linux, use the unix format and unzip the files in linux.


V1.5 17/07/2012

add back the web_root parameter to set the URL base.


V1.4 17/07/2012

Have uploaded the latest version. 
changes
Removed support for a movie.downlaoded api call that was only used in a seperate branch and is not expected to be merged.
Modified the passthrough to allow a manual call to this script (i.e. does not need to be called from SABnzbd).
Have added a helpfile that explains the setup options in a bit more detail.
Modified the .cfg.sample file to use 60 as a default delay and now specify that 60 should be your minimum to ensure the renamer.scan finds newly extracted movies.


V1.3 and earlier were not fully tracked, as the script itself (not files) was posted on the QNAP forums.