Parallel Computing is the art of running computations more or less simultaneously in a multi-cores or multi-processor environment. Large problems can often be divided into smaller ones, which can then be solved at the same time. To distribute a problem to several processors requires great deal of management
Distributed computing is when we have a system of computers networked together to solve a common problem. The computers are placed in different locations where each computer can be seen as a node. The distributed nodes may all be placed in the same room or they may be placed in separate locations far from each others. The computers may also be any combination of two previously mentioned cases. In a node, the computer casings just need to be separate, where the computers work on solving a common problem and then we have distributed computing. With distributed computing one strives to obtain scalability with the nodes in administration, size, functionality and geography. Two good examples of distributed computing systems are automatic teller machines and big data clusters.
Cloud computing is shared pools of configurable computer system resources and higher-level services that can be rapidly provisioned with minimal management effort, often over the Internet. It is not unusual for cloud computing services use techniques of distributed computing. Three very popular cloud services from corporate giants as AWS from Amazon, Azure from Microsoft and Google Cloud from Google. Cloud computing have become very popular lately where customers supposedly rent resources at reduced cost compared to owning and maintaining the equipment oneself. Other important factors for an organization to use cloud computing services are such as uptime reliability, reliability of maintenance and speed of service. Legal regulations or internal policies may also force an organization use cloud computing to reduce harm from equipment failure, theft, burn outs, fires, natural disasters and catastrophes.
High performance computing (HPC) or supercomputing is the class of computing for which performance or alternately time to solution is the primary property. HPC integrates mathematics and technology together. A distributed network of computers can be used as a HPC solution. If the distributed network of computers are located in the same room then possibly we have a supercomputer.
To be good at high-performance computing, it is required one it good at mathematical modeling and numerical analysis. Further, one should also understand how to exploit the hardware to its fullest potential. High-performance computing deals with both computational science and computationel engineering. To be a good HPC scientist, it is required one is able to: (1) Represent a real-world problem and its actions into mathematical formulation or model, (2) simplify and approximate the mathematical model (1) into a new model for use in computers. (3) Present an algorithm as a solver to the approximate problem (2), (4) study the stability and rate of convergence of the solver (3) to finally (5) compute, evaluate and validate the solution to the approximate problem (2).
The Computer Side To get a computer enginer, combine a electrical engineer with a computer scientist. To get a computer scientist, remove some hardware skills from the computer engineer, then add in more software skills. To get a computational scientist, combine a Mathematician with a computer scientist. To get a computational engineer, combine a mathematician with a computer engineer. The Data Side To get a data scientist, combine a statistician with a computer scientist. Also add in machine learning. To get a data engineer, combine a statistician with a computer engineer with emphasis on the computer engineer. To get a data analyst, combine a economist with a computer scientist. The HPC Side Parallel Computing, is using several processors and cores in a single machine. Distributed Computing, is using several computers to solve a common goal. Supercomputing when Distributed Computing is in a single location. Cloud Computing, is when using one or several computers remotely.
Prefered applications are in banking and finance. A combination of computational scientist and data scientist with stronger lean towards computational scientist. For skills in distributed computing, add in a few necessary computational engineer skills. For increased performance and stability, skip Python and add in C/C++.