Notes

• ​If you can solve a problem with a simple heuristic, do that. Sometimes you don't need machine learning.​

• Data is not always useful and it doesn't matter how much of it you have.

  There’s no mathematical tool to tell you if your hypothesis is true; you can only see whether it is consistent with the data, and if the data is sparse or unclear, your conclusions are uncertain.

• ​By writing the data generating process first, and therefore knowing parameters, you can get more confidence that your model is well constructed.​

Links

• ​Some advice for young and aspiring Data Scientists​

• ​A Beginner’s Guide to Data Engineering — Part I​

• ​The Rise of the Data Engineer​

• ​Cookiecutter Data Science - Logical, reasonably standardized, but flexible project structure for doing and sharing data science work. (Code)

• ​Best way to organize research code? (2018)​

• ​Data Science Cheat Sheet​

• ​Our world in data​

• ​Free data science books​

• ​Pachyderm - Reproducible Data Science at Scale.

• ​Data Science Cheat Sheets​

• ​Data Science in Visual Studio Code using Neuron, a new VS Code extension (2018)​

• ​Virgilio - Mentor for Data Science E-Learning.

• ​Awesome Data Science with Python - Curated list of Python resources for data science.

• ​Data Science (Cheat Sheets)​

• ​nteract - Interactive computing suite for you.

• ​120 Data Science Interview Questions​

• ​How To Become a Data Engineer​

• ​Ingestion Data Mapping Language​

• ​The reference implementation of IDML for the JVM​

• ​Pandas - Powerful Python data analysis toolkit.

• ​Programming Language Support for Data-intensive Applications meeting (2019)​

• ​Datasette - Tool for exploring and publishing data. (datasette-graphql)

• ​Weld - High-performance runtime for data analytics applications.

• ​Vaex - Out-of-Core DataFrames for Python, visualize and explore big tabular data at a billion rows per second.

• ​PyParis 2018 - Vaex: Out of Core Dataframes for Python​

• ​Maarten Breddels & Jovan Veljanoski- A new approach to DataFrames and pipelines - PyData London 2019 (GitHub)

• ​The Data Engineering Cookbook​

• ​Data science without borders - Wes McKinney (2017)​

• ​Curated list of awesome ETL frameworks, libraries, and software​

• ​Ibis - Python data analysis framework for Hadoop and SQL engines.

• ​Kyso - Data analytics knowledge hub.

• ​Feather - Fast, interoperable binary data frame storage for Python, R, and more powered by Apache Arrow.

• ​ROOT system - Provides a set of OO frameworks with all the functionality needed to handle and analyze large amounts of data in a very efficient way.

• ​Redash - Make Your Company Data Driven. Connect to any data source, easily visualize, dashboard and share your data. (Web)

• ​Prefect - New workflow management system, designed for modern infrastructure and powered by the open-source Prefect Core workflow engine.

• ​Turn Python Scripts into Beautiful ML Tools (2019) (HN)

• ​Foundations of Data Science (2019) (HN)

• ​Gyana - No code desktop data science tool. (Article)

• ​Numba - NumPy aware dynamic Python compiler using LLVM. (5 minute guide)

• ​What's your typical data pipeline in a small company? (2019)​

• ​dbt - Data build tool. Analytics engineering workflow.

• ​Apache Airflow - Platform to programmatically author, schedule, and monitor workflows. (Tutorial)

• ​Overview of Popular Open Source Big Data Technologies (2018)​

• ​Introducing Apache Arrow Flight: A Framework for Fast Data Transport (2019)​

• ​Understanding Apache Arrow Flight (2019)​

• ​New Developments in the Open Source Ecosystem: Apache Spark 3 0, Delta Lake, and Koalas (2019)​

• ​Materials and IPython notebooks for Python for Data Analysis, 2nd Edition book​

• ​Technical Notes On Using Data Science & Artificial Intelligence​

• ​Amazon Data Science Interview (2018)​

• ​Data Analysis and Prediction Algorithms with R (2019)​

• ​Elegant SciPy book​

• ​Data science blogs​

• ​Awesome Data Science​

• ​Things About Real-World Data Science Not Discussed In MOOCs and Thought Pieces (2018)​

• ​Apache Zeppelin - Web-based notebook that enables data-driven, interactive data analytics and collaborative documents with SQL, Scala and more.

• ​Apache Nifi - Easy to use, powerful, and reliable system to process and distribute data.

• ​Data Science: Principles and Practice Course materials (2018/19)​

• ​Modern Data Practice and the SQL Tradition (2019) (HN)

• ​Koalas - Pandas API on Apache Spark.

• ​Ask HN: What does your BI stack look like? (2019)​

• ​Easy Data Transform - Transform Your Data Without Programming. (HN)

• ​Prophet - Tool for producing high quality forecasts for time series data that has multiple seasonality with linear or non-linear growth.

• ​Dagster - Python library for building data applications: ETL, ML, Data Pipelines, and more.

• ​CuPy - NumPy-like API accelerated with CUDA.

• ​An Introduction To Data Science On The Linux Command Line (2019) (HN)

• ​How to analyse 100 GB of data on your laptop with Python (2019)​

• ​Metaflow - Framework for real-life data science. (HN) (Code)

• ​SaturnCloud - Manage Data Science applications so Data Scientists don't have to do DevOps.

• ​Falcon - Interactive Visual Analysis for Big Data.

• ​Materialize - Streaming Data Warehouse. (Web)

• ​160+ Data Science Interview Questions​

• ​Data Science Interview Questions​

• ​Google Cloud DataLab - Interactive tools and developer experiences for Big Data on Google Cloud Platform.

• ​Great Expectations - Leading tool for validating, documenting, and profiling, your data to maintain quality and improve communication between teams. (Code)

• ​Path to a free self-taught education in Data Science​

• ​Common Workflow Language - Open standard for describing analysis workflows and tools. (HN)

• ​Apache Kudu - Completes Hadoop's storage layer to enable fast analytics on fast data.

• ​Time Series Forecasting Best Practices & Examples​

• ​Turing Way - Lightly opinionated guide to reproducible data science. (Code)

• ​What to do when you didn’t get any medal in a Kaggle competition? (2020)​

• ​Build a Career in Data Science book (2020)​

• ​Data Science Tutorials in Julia (Code)

• ​Data Science Resources​

• ​Master Data Analysis with Python​

• ​Introduction to Data Science book (2020) (Code)

• ​VisiData - Terminal spreadsheet multitool for discovering and arranging data. (Code)

• ​Sisu - Fastest Diagnostic Platform for Structured Data. (Introducing Sisu)

• ​Deepnote - Data science notebook for teams. (Docs)

• ​Towards Data Science blog​

• ​Scaling Pandas: Dask vs Ray vs Modin vs Vaex vs RAPIDS (2020) (HN)

• ​A graphical analysis of women's tops sold on Goodwill's website (HN)

• ​1.1B Taxi Rides Using OmniSciDB and a MacBook Pro (2020) (HN)

• ​Data Science Interview Resources​

• ​Data Science Meets Devops: MLOps with Jupyter, Git, & Kubernetes (2020)​

• ​dplyr - Grammar of data manipulation, providing a consistent set of verbs that help you solve the most common data manipulation challenges. (Web)

• ​dstack.ai - Open-source tool for building data and ML applications using Python and R. (HN)