emily.space / ocfinder
[READ-ONLY] Mirror of https://github.com/emilyhunt/ocfinder. Code for my PhD for finding open clusters, neatly organised.
Star
Fork 0
Watch

Commits on master

Fix to UPMASK covariance determination if no covariances are in the data. Previously, no error information was being used at all! 0.1.0 0.1.0-alpha cac20 master
emilyhunt committed
Fix to UPMASK covariance determination if no covariances are in the data. Previously, no error information was being used at all!
emilyhunt committed
Extra UPMASK mode.
emilyhunt committed
A little bit of extra UPMASK goodies. - Added a new cut_mode that uses r_50 instead of r_t, since the former is likely to be a lot more accurate for CAC20 OCs - A little bugfix to parameter writing.
emilyhunt committed
Got UPMASK up and running! - After a few small bugfixes, imported notebook code was easy to get going! - An UPMASKPipeline was added. Beware that it works in some odd ways because it subclasses ClusteringAlgorithm while not behaving like one really... - Still have hilariously annoying issues with getting verbosity to turn itself on or off - Maybe cut parameters should be user-changeable opposed to hard coded? Would be nice to let me run with different ones. This will potentially be quite necessary because UPMASK is so, so sensitive to the way the data are cut a-priori
emilyhunt committed
Little bugfix in the _blank_statistics_dataframe caused by a wayward space.
emilyhunt committed
Minor change to default PostProcessor settings.
emilyhunt committed
Small blank statistics DataFrame bugfix - I forgot to include a few columns with field etc IDs!
emilyhunt committed
Updated the blank statistics DataFrame to now be compatible with changes to ocelot.calculate.
emilyhunt committed
Re-factored the radius cut based on notebook 0_gmm_parameter_tuning section 4. - The radius cut now doesn't have a "max distance" and is instead defined with a minimum angular size that's always deemed real. - The radius cut now works with the new GMMPostProcessor pipeline and has been added as a default option.
emilyhunt committed
Bugfix to run_gmm(), since DataFrame elements cannot be lists....
emilyhunt committed
The postprocessor now also saves the current parallax range to the mixture parameters, when available!
emilyhunt committed
run_gmm() now also saves the current parallax range to the mixture parameters.
emilyhunt committed
GMMPostProcessor() now tells you the real field/run name of whatever it's working on, making restarts and bugfixing easier.
emilyhunt committed
Bugfix to the GMMPostProcessor(). - We now require the sum of all membership probabilities of a cluster to be larger than some very small number (1e-100), which prevents later divsions by zero when taking weighted mean parameters.
emilyhunt committed
Bugfix to the GMMPostProcessor(). - Numpy arrays don't have a .index method, which we need! Changed this function so that 'names' is cast as a list in __init__().
emilyhunt committed
Bugfix to run_gmm(). - We shouldn't crash anymore in the unlikely event that a mixture has no member stars.
emilyhunt committed
Added a current partition count to run_gmm() to help with debugging.
emilyhunt committed
Many commits bundled into one due to my .gitignore being wrong and accidentally uploading a LOT of unnecessary trash... this is why you should push after every commit!!!
emilyhunt committed
run_dbscan bugfix. - Added a min_cluster_size (defaults to 5) below which to drop any clusters.
emilyhunt committed
ClusteringAlgorithm _get_clusterer_args() bugfix. - Won't open with attribute .values anymore for non-feather files (only feathers require dropping columns like this)
emilyhunt committed
Pipeline _open() method update. - Added support for loading sparse matrices, which also have a .npz file extension but have a specific signature.
emilyhunt committed
DBSCANPreprocessor plotting bugfix. - show_figure=False added to plotting call to prevent brief matplotlib window flashes (which causes a crash if the figure can't be shown at all due to SSH'ing)
emilyhunt committed
Start functionality for the DBSCANPreprocessor in response to an ocelot bugfix and a need to get going again. - Added a "start" in the apply() method of DBSCANPreprocessor.
emilyhunt committed
Bugfix to ClusteringAlgorithm superclass _save_clustering_result. - Previously, empty clustering results would cause an error because pandas couldn't concatenate an empty list of DataFrames. - There's now a default blank DataFrame (with all requisite column names) that takes its place in this case. - Write better unit tests ;) - Also added a "start" in the apply() method of ClusteringAlgorithm.
emilyhunt committed
Bugfix to HDBSCAN clustering. - I was requiring "epsilon" as an input (a-la DBSCAN) in error.
emilyhunt committed
Added sorting to Pipeline when only a directory is specified as the input. Otherwise stuff can end up very messy and out of sync!
emilyhunt committed
Changed ClusteringAlgorithm's __init__ kwarg "calculate_cluster_stats" to "maximum_cluster_size_to_save". Clusters larger than this will never have baby data gaia views saved for them, saving on some memory use if the entire field gets selected as a cluster.
emilyhunt committed
DBSCAN preprocessing small change. - Now also recording the standard deviation of ACG epsilon runs by default.
emilyhunt committed
DBSCAN work and Preprocessor HEALPix support. - Re-factored dbscan_preprocessing to DBSCANPreprocessor, which uses the Pipeline superclass. Has been validated! - Made and validated DBSCANPipeline class that subclasses ClusteringAlgorithm. - Updated OCFinder's preprocessor to work with HEALPix IDs, building on a little ocelot addition I also made today.
emilyhunt committed
Result plotting bugfixes! - Fixed various bugs in result plotting. - Made the title formatting a little neater and not cut off.
Emily Hunt committed
HDBSCANPipeline bugs and result plotting! - Sorted out lots of HDBSCANPipeline bugs and ClusteringAlgorithm bugs. - It now runs and makes good results!!! Yay!!! - Implemented a ResultPlotter on the pipeline. It still needs a tiny bit more bugfixing, but is nearly there!
Emily Hunt committed
GMMs and superclasses! - Added stuff for GMMs from the prototyping Jupyter notebook - Added a Pipeline superclass and made other stuff subclass it. - Also added a ClusteringAlgorithm superclass that subclasses Pipeline with some extra info. - All together, all of the stuff is super, super powerful! - Will need a lot of testing! But the import runs at least... lol - Will also need a GMM cutting suite/pipeline, at the moment it just outputs stuff as is.
Emily Hunt committed
Bits of new(ish) running code. - Finished DBSCAN processing code and added a new run_dbscan. - Started working on HDBSCAN running code.
Emily Hunt committed
A mix of notebook ports and bits of new(ish) running code. - Imported UPMASK running code - Made a function to handle en-masse pre-processing - Made a function to save clustering results - Started work on a DBSCAN preprocessing/epsilon function
Emily Hunt committed
Initial commit 2: Electric Boogaloo
Emily Hunt committed
Initial commit
Emily Hunt committed