Machine Learning
Today, Let me share a list of website related to Machine Learning.This list is created by referring to dhilipsiva@gmail.com 's e-mails to BangML meetup group. Acknowledged.
- CycleGAN: Software that generates photos from paintings, turns horses into zebras, performs style transfer, and more (from UC Berkeley): https://github.com/junyanz/CycleGAN
- weld: a runtime and language for accelerating data analytics frameworks: https://github.com/weld-project/weld
- deform-conv: D eformable Convolution in TensorFlow / Keras: https://github.com/felixlaumon/deform-conv
- sact: Spatially Adaptive Computation Time for Residual Networks: https://github.com/mfigurnov/sact
- CLR: Cyclical Learning Rate: https://github.com/bckenstler/CLR
- seq2seq: A general-purpose encoder-decoder framework for Tensorflow: https://github.com/google/seq2seq
- deep-photo-styletransfer: Code and data for paper "Deep Photo Style Transfer": https://github.com/luanfujun/deep-photo-styletransfer
- DiscoGAN-pytorch: PyTorch implementation of "Learning to Discover Cross-Domain Relations with Generative Adversarial Networks": https://github.com/carpedm20/DiscoGAN-pytorch
- evolution-strategies-starter: Starter code for Evolution Strategies: https://github.com/openai/evolution-strategies-starter
- AdaIN-style: Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization: https://github.com/xunhuang1995/AdaIN-style
- TC-Bot: User Simulation for Task-Completion Dialogues: https://github.com/MiuLab/TC-Bot
- e2e-model-learning: Task-based end-to-end model learning: https://github.com/locuslab/e2e-model-learning
- csgm: Code to reproduce results from the paper: "Compressed Sensing using Generative Models": https://github.com/AshishBora/csgm
- rnnprop: RNNprop: https://github.com/vfleaking/rnnprop
- DeepSketchHashing: Deep Sketch Hashing: https://github.com/ymcidence/DeepSketchHashing
- tensorflow/models: Learning to Remember Rare Events: https://github.com/tensorflow/models/tree/master/learning_to_remember_rare_events
- ORB_SLAM2: Real-Time SLAM for Monocular, Stereo and RGB-D Cameras, with Loop Detection and Relocalization Capabilities: https://github.com/raulmur/ORB_SLAM2
- rllab: is a framework for developing and evaluating reinforcement learning algorithms, fully compatible with OpenAI Gym: https://github.com/openai/rllab
- voxnet: 3D/Volumetric Convolutional Neural Networks with Theano+Lasagne: https://github.com/dimatura/voxnet
- dynamics: A Compositional Object-Based Approach to Learning Physical Dynamics: https://github.com/mbchang/dynamics
- CommAI-env - A platform for developing AI systems as described in A Roadmap towards Machine Intelligence: https://github.com/facebookresearch/CommAI-env
- rwa - Machine Learning on Sequential Data Using a Recurrent Weighted Average: https://github.com/jostmey/rwa
- MazeBase - Simple environment for creating very simple 2D games and training neural network models to perform tasks within them: https://github.com/facebook/MazeBase
- ontology - The Audio Set Ontology aims to provide a comprehensive set of categories to describe sound events: https://github.com/audioset/ontology
- mv3d - Multi-view 3D Models from Single Images with a Convolutional Network: https://github.com/mtatarchenko/mv3d
- universal - Universal adversarial perturbations: https://github.com/LTS4/universal
- prophet - Tool for producing high quality forecasts for time series data that has multiple seasonality with linear or non-linear growth: https://github.com/facebookincubator/prophet
- faiss - A library for efficient similarity search and clustering of dense vectors: https://github.com/facebookresearch/faiss
- CausalImpact - An R package for causal inference in time series: https://github.com/google/CausalImpact
- GPflow - Gaussian processes in TensorFlow: https://github.com/GPflow/GPflow/
- optnet - Differentiable Optimization as a Layer in Neural Networks: https://github.com/locuslab/optnet
- RNN-based generative models for speech: https://github.com/sotelo/parrot
- An SSBM player based on Deep Reinforcement Learning: https://github.com/vladfi1/phillip
- Shake-Shake regularization of 3-branch residual networks: https://github.com/xgastaldi/shake-shake
- Benchmarking State-of-the-Art Deep Learning Software Tools: https://github.com/hclhkbu/dlbench
- Public repo for helpful scripts when using the YouTube Bounding Boxes dataset: https://github.com/mbuckler/youtube-bb
- AirSim - Open source simulator based on Unreal Engine for autonomous vehicles from Microsoft AI & Research: https://github.com/Microsoft/AirSim
- dlib - A toolkit for making real world machine learning and data analysis applications in C++: https://github.com/davisking/dlib/
- Cyclades - This repository contains code for Cyclades, a general framework for parallelizing stochastic optimization algorithms in a shared memory setting: https://github.com/amplab/cyclades
- Deep learning with dynamic computation graphs in TensorFlow: https://github.com/tensorflow/fold
- Starter code for working with the YouTube-8M dataset: https://github.com/google/youtube-8m
- Vid2speech: Speech Reconstruction from Silent Video: https://github.com/arielephrat/vid2speech
- PatchBatch - a Batch Augmented Loss for Optical Flow: https://github.com/DediGadot/PatchBatch
- Wasserstein GAN: https://github.com/martinarjovsky/WassersteinGAN
- Keras Implementation of Google's Inception-V4 Architecture: https://github.com/kentsommer/keras-inceptionV4
- Implementation of Meta-RL A3C algorithm: https://github.com/awjuliani/Meta-RL
- DeepPose implementation on TensorFlow: https://github.com/asanakoy/deeppose_tf
- Exact Soft Confidence-Weighted Learning: https://github.com/IshitaTakeshi/SoftConfidenceWeighted.jl
- DyNet: The Dynamic Neural Network Toolkit: https://github.com/clab/dynet
- ENet Training: https://github.com/e-lab/ENet-training
- Generic Foreground Segmentation in Images: https://github.com/suyogduttjain/pixelobjectness
- Plexus - Interactive Emotion Visualization based on Social Media: https://github.com/xavierwu2016/plexus
- Code for Residual and Plain Convolutional Neural Networks for 3D Brain MRI Classification paper: https://github.com/neuro-ml/resnet_cnn_mri_adni
- Image Compression with Neural Networks: https://github.com/tensorflow/models/tree/master/compression
- A library for probabilistic modeling, inference, and criticism. Deep generative models, variational inference: https://github.com/blei-lab/edward
- KOC University deep learning framework: https://github.com/denizyuret/Knet.jl
- Domain Transfer Network. Tensorflow implementation of unsupervised cross-domain image generation: https://github.com/yunjey/dtn-tensorflow
- Improving Convolutional Networks via Attention Transfer: https://github.com/szagoruyko/attention-transfer
- Open-Source Neural Machine Translation in Torch: https://github.com/OpenNMT/OpenNMT
- Texture Networks: Feed-forward Synthesis of Textures and Stylized Images: https://github.com/DmitryUlyanov/texture_nets
- The Predictron: End-To-End Learning and Planning: https://github.com/zhongwen/predictron
- Deluge Networks: https://github.com/xternalz/DelugeNets
- Learning from Simulated and Unsupervised Images through Adversarial Training (TensorFlow): https://github.com/carpedm20/simulated-unsupervised-tensorflow
- FastMask: Segment Multi-scale Object Candidates in One Shot: https://github.com/voidrank/FastMask
- Visual7W visual question answering models: https://github.com/yukezhu/visual7w-qa-models
- Language Modeling with Gated Convolutional Networks: https://github.com/DingKe/nn_playground/tree/master/gcnn
- Semantic JPEG image compression using deep convolutional neural network (CNN): https://github.com/iamaaditya/image-compression-cnn
- TextBoxes: A Fast Text Detector with a Single Deep Neural Network: https://github.com/MhLiao/TextBoxes
- Implementation of video captioning from the paper "Temporal Tessellation for Video Annotation and Summarization": https://github.com/dot27/temporal-tessellation
- TFLearn - Deep learning library featuring a higher-level API for TensorFlow: http://tflearn.org
C++
Today Let's revise / refresh few aspects of C++ object oriented programming language
- A class has implicit members
- Default constructor
- Copy constructor. It cannot take object as argument. It always takes object reference as argument.
- Assignment operator
- Default destructor
- Now some specific points about constructor
- A class can have default constructor and explicit constructor. Explicit constructor force compiler for not to do implicit conversion
- However a constructor cannot be virtual
- The copy constructor is called in following cases
- If function return a object by value
- Function argument is object by value
- When we construct a object using another object
- Many times compiler generates temporary object
- Storage class
- auto
- register. Here '&' operator cannot be used
- static. The variable is initialize with value 0
- extern The variable is initialize with value 0
- Storage qualifier OR type qualifier
- volatile
- mutable
- restrict C99 standard. compiler optimization that avoid multiple loading of same variable.
- _Atomic . C99 standard
- const . In the context of pointer, const can be used 3 different ways
- int const * const x
- int const * x
- int * const x
- As we know for member function of class "this" pointer is passed by default. It is always passed as constant pointer.
- The following operator cannot be overloaded
- sizeof
- "."
- ".*"
- typeid
- "::"
- "?:"
- alignof
- Now about type casting of variables
- dynamic_cast . Runtime type check. It is used to case
- base class pointer to derived class pointer
- derive class pointer to another derive class pointer of same base class as parent.
- reinterpret_cast : cast to unrelated object
- static cast. compile time type check. for related object
- const_cast
- duration cast from chrono library
- C++ has specific features like
- Class
- Inline function
- Default argument
- Exception
- Namespace
- Boolean type
- Object Oriented Programming features
- Abstraction
- Encapsulation (information hiding)
- Inheritance
- Polymorphism
- Something about Inheritance
- Virtual inheritance is used to avoid diamond problem
- Inheritance applications are (1) code reuse and (2) overriding base class implementaiton
- 5 types of inheritance
- Single
- Multiple
- Multilevel
- Hierarchical
- Hybrid
- Something about polymorphism
- Static polymorphism
- Function overloading is also called ad-hoc polymorphism
- Templates is also called parametric polymorphism
- Dynamic polymorphism
- Subtyping is also called inclusion polymorphism
- C++ implements virtual table with single dispatch only. No multiple dispatch
Service-Oriented Architecture
Service-oriented architecture is a buzzword. It is all about 1. Loose coupling 2. Modular programme 3. connected components using various protocols over IP. SOA is the philosophy of encapsulating application logic in services with a uniformly defined interface and making these publicly available via discovery mechanism. Let's look at some more aspects about SOA
Core Values
Core Values
- More focus on 'business value' compare to 'technical strategy'
- So more focus on 'strategic goal' compare to 'project benifits'
- The components should have 'intrinsic inter-operability'
- More importance to 'shared services' compare to 'implementation for specific purpose'
- Flexibility is more important compare to optimisation
- Here, 'Evolution refinement' is more important compare to 'initial perfection'
SOA is similar to 1. Modular programming 2. Distributed computing 3. mashups 4. SaaS and 5. Cloud computing. Still it is different. SOA has 11 principals.
1. The service contract should be standardized. That includes functional expressions may be using WSDL. Data model may be using XML schema and 3. Policy document.
2. Reference autonomy standardization : The other components are merely aware about presence of other service component. It fosters the loose coupling
3. Service Location Transparency : Service can be executed, deployed anywhere.
4. Service Longevity : Service should run for ever. It also means backward compatible implementation.
5. Service Abstraction means the service is jut a black box. (1) The functional aspects of service (2) Technical information (3) Logic / Algorithm and (2) Quality of service, all these are hidden.
6. Service autonomy : It means, service should be more predictable, reliable and less depends on external environment. Run time service autonomy achieved by dedicated CPU and memory so external environment has lesser impact. Design time service autonomy means, service can be evolve and re-design without impacting consumers. More applicable to entity service and utility service.
7. Service statelessness: The service is implemented as stateless. However it can take support for other system to maintain state. Service may need to maintain state about (1) Context data (2) Business data and (3) Session data.
8. Service granularity define the scope of service. It impacts performance and message size for the service.
9. Service normalization is trade off between redundancy and performance
10. Service composability A service can play two role here (1) Service controller and (2) Service member
11. Service discovery : Using service discovery protocol
12. Service re-usability : The service, which is part of multiple business processes is more reusable. The service with extra functionality will be reusable in future.
13. Service encapsulation: The service not part of SOA, initially.
Each SOA building block plays any of the three roles
1. Service provider
2. Service broker = service registry = service repository
3. Service consumer = service requester
Implementation
SOA can be implemented in multiple ways
1. SOA can be implemented using any service based technology. It can be web services using SOAP+WSDL or RESTful protocol. It can be using (1) Jini and JBI (2) CORBA (3) REST (4) WCF
2. Messaging technology can be used (1) ActiveMQ (2) JMS (3) RabbitMQ
3. OPC-UA is a M2M protocol. It can be web service oriented. It can be binary protocol over TCP/IP. It can by hybrid approach of binary protocol over SOAP.
4. Apache Thrift is a RPC based framework by FaceBook. It provides IDL and binary protocol.
5. SORCER is Java based distributed computing platform, for service oriented computing environment.
6. The programming language like C# .Net, Java, BPEL also has support for SOA
7. WS-CDL and WS-Coordination Specifications extend SOA further for workflows and business process.
Service Oriented Modelling
In SOA framework, Service Oriented Modelling identifies various disciplines to conceptualize, analyze, design, and architect service-oriented assets. Here three approaches are identified
1. SDDM: Service-oriented design and development methodology
2. SOMA: Service-oriented modeling and architecture
3. SOMF: Service-oriented modeling framework. It is about what to do.
Extension and variants
SOA can be further extended with various variants
1. SOA 2.0 is combination of SOA and EDA. Event Driven Architecture. It has three components. (1) Event emitters (agents) (2) Event consumers (sinks OR event processing engine) (3) Event channel. It can be (3.1) message oriented middleware or (2) TCP/IP link (3) File: XML, flat file, or even e-mail.
2. Web 2.0 has same philosophy as SOA, but it serves different user needs.
3. Microservices : (1) Fine grained services may be a process of OSGi Bundle (2) Light weight protocol may be HTTP or shared memory. Results in continuous software development process. Jolie programming language allows to think and program in terms of microservices. The complexity moves to network as (1) network latency (2) message format (3) load balancing and (4) fault tolerance
I hope, the readers like this overview of SOA.
I hope, the readers like this overview of SOA.
Database Key
Today one new label is added. "Database". There are set of articles planned in this category. Today is the first one.
Most of us know, in RDBMS, database table has primary key and foreign key. Any idea about other types of key? Below tables describes all database keys and relations among them.
Most of us know, in RDBMS, database table has primary key and foreign key. Any idea about other types of key? Below tables describes all database keys and relations among them.
Primary
|
Unique
|
Surrogate
|
Foreign
|
Composite
|
Simple
|
||
1
|
Primary
|
1
|
1
|
May be
|
May be
|
May be
|
May be
|
2
|
Unique
|
May be
|
1
|
May be
|
May be
|
May be
|
May be
|
3
|
Surrogate
|
May be
|
1
|
1
|
May be
|
0
|
1
|
4
|
Foreign
|
0
|
May be
|
0
|
1
|
May be
|
May be
|
5
|
Composite
|
May be
|
May be
|
0
|
May be
|
1
|
0
|
6
|
Simple
|
May be
|
May be
|
0
|
May be
|
0
|
1
|
7
|
Super
|
0
|
1
|
0
|
0
|
May be
|
May be
|
8
|
Minimal super
|
May be
|
1
|
0
|
0
|
May be
|
May be
|
9
|
Tirival Super
|
May be
|
1
|
0
|
May be
|
1
|
0
|
10
|
Candidate
|
May be
|
1
|
May be
|
May be
|
May be
|
May be
|
11
|
Alternate
|
0
|
1
|
1
|
May be
|
May be
|
May be
|
12
|
Natural
|
May be
|
1
|
0
|
May be
|
1
|
| Super | Minaml super | Tirival Super | Candidate | Alternate | Natural | ||
| 1 | Primary | May be | May be | 0 | 1 | 0 | May be |
| 2 | Unique | 1 | May be | May be | 1 | May be | May be |
| 3 | Surrogate | 0 | 0 | 0 | May be | May be | 0 |
| 4 | Foreign | May be | May be | 0 | 0 | 0 | 0 |
| 5 | Composite | May be | May be | 1 | 1 | May be | 0 |
| 6 | Simple | 0 | May be | 0 | May be | May be | May be |
| 7 | Super | 1 | May be | 1 | 0 | 0 | |
| 8 | Minimal super | 1 | 1 | 0 | 1 | May be | May be |
| 9 | Tirival Super | 1 | 0 | 1 | 1 | May be | 0 |
| 10 | Candidate | 1 | 1 | 1 | 1 | May be | May be |
| 11 | Alternate | 1 | 1 | 0 | 1 | 1 | May be |
| 12 | Natural | 0 | 1 | May be | 1 |
The above table should be read, row wise
For example 1st row:
Is primary key is unique? 1 = Yes
Is it surrogate? The surrogate key can become primary key. However the primary key is not always surrogate. So answer "May be"
and so on. Here 1 = Yes and 0 = No
Here are some key points about database keys
- The primary key cannot be NULL
- The unique key can be NULL
- The primary key is always unique
- All unique keys are candidate key to become primary key
- Primary key can be one of the candidate key or it can be surrogate key
- The candidate key, who is not primary key is alternate key.
- If surrogate key is primary key, then all candidate keys are alternate keys.
- Simple key and composite key are mutually exclusive.
- The compound key should have at least two simple keys
- The composite key is extension of compound key
- Natural key and domain key are synonyms.
- Composite key, compound key, aggregate key and concatenated key. All are in a way same.
- Minimal super key and candidate key are in a way same.
- Trivial super key means all columns.
Stay tune to this blog "Express YourSelf !" for more interesting articles.
Internet of Things
I have posted few mindmaps on this blog: IPSec , IMS and SIP, ANDSF
Today one more. It is about IoT. "Internet of Things" is a buzzword. In fact, IoT and 5G are like two sides of single coin. There are many standards related to IoT defined by various standarisation bodies. These standards are for wired connection and wireless connection. The wireless connection is also referred as Low Power Wide Area Network (LP-WAN). It is further categorizes licensed spectrum and Non-licensed spectrum. IoT is also about many protocols used. The IoT application uses database with specific set of features.
As per Wikipedia
A mind map is a diagram used to visually organize information. A mind map is hierarchical and shows relationships among pieces of the whole.[1] It is often created around a single concept, drawn as an image in the center of a blank page, to which associated representations of ideas such as images, words and parts of words are added. Major ideas are connected directly to the central concept, and other ideas branch out from those.
Today one more. It is about IoT. "Internet of Things" is a buzzword. In fact, IoT and 5G are like two sides of single coin. There are many standards related to IoT defined by various standarisation bodies. These standards are for wired connection and wireless connection. The wireless connection is also referred as Low Power Wide Area Network (LP-WAN). It is further categorizes licensed spectrum and Non-licensed spectrum. IoT is also about many protocols used. The IoT application uses database with specific set of features.
As per Wikipedia
A mind map is a diagram used to visually organize information. A mind map is hierarchical and shows relationships among pieces of the whole.[1] It is often created around a single concept, drawn as an image in the center of a blank page, to which associated representations of ideas such as images, words and parts of words are added. Major ideas are connected directly to the central concept, and other ideas branch out from those.
