Articles

Execute Spark Applications on Databricks Using the REST API

,

Introduction

While many of us are habituated to executing Spark applications using the ‘spark-submit’ command, with the popularity of Databricks, this seemingly easy activity is getting relegated to the background. Databricks has made it very easy to provision Spark-enabled VMs on the two most popular cloud platforms, namely AWS and Azure. A couple of weeks ago, Databricks announced their availability on GCP as well. The beauty of the Databricks platform is that they have made it very easy to become a part of their platform. While Spark application development will continue to have its challenges – depending on the problem being addressed – the Databricks platform has taken out the pain of having to establish and manage your own Spark cluster.

Using Databricks

Once registered on the platform, the Databricks platform allows us to define a cluster of one or more VMs, with configurable RAM and executor specifications. We can also define a cluster that can launch a minimum number of VMs at startup and then scale to a maximum number of VMs as required. After defining the cluster, we have to define jobs and notebooks. Notebooks contain the actual code executed on the cluster. We need to assign notebooks to jobs as the Databricks cluster executes jobs (and not Notebooks). Databricks also allows us to setup the cluster such that it can download additional JARs and/or Python packages during cluster startup. We can also upload and install our own packages (I used a Python wheel).

Source de l’article sur DZONE

/par

Why Python Is Best for Machine Learning

,

Today, most companies are using Python for AI and Machine Learning. With predictive analytics and pattern recognition becoming more popular than ever, Python development services are a priority for high-scale enterprises and startups. Python developers are in high-demand — mostly because of what can be achieved with the language. AI programming languages need to be powerful, scalable, and readable. Python code delivers on all three.

While there are other technology stacks available for AI-based projects, Python has turned out to be the best programming language for this purpose. It offers great libraries and frameworks for AI and Machine Learning (ML), as well as computational capabilities, statistical calculations, scientific computing, and much more. 

Source de l’article sur DZONE

/par

Build a REST API with Node.js and HarperDB

,

If you are building an application using Node.js, it can get a little overwhelming since there are a variety of databases to choose from and different ways to build APIs. One way to reduce development time and focus on the problem you are trying to solve is to use a Database as a service to store the data. The advantage of this approach is to use a cloud database system without purchasing hardware which can be cost and time-effective.

One such database service is HarperDB Cloud. To build REST APIs rapidly this service allows us to perform all database operations using a single endpoint. It supports a variety of programming languages such as JavaScript, Java, Python, and so on. Some of the features of HarperDB are the following:

Source de l’article sur DZONE

/par

Configuring REST Channels for CORS in Zato

,

With the rise of Single-Page Applications (SPA) in web frontends, it is often the case that backend REST APIs based on Zato need to be configured for CORS. This article will explore what CORS is and how to make Zato participate in scenarios using it.

Terminology

CORS, as an acronym, has several parts:

Source de l’article sur DZONE

/par

Big / Bug Data: Analyzing the Apache Flink Source Code

,

Applications used in the field of Big Data process huge amounts of information, and this often happens in real time. Naturally, such applications must be highly reliable so that no error in the code can interfere with data processing. To achieve high reliability, one needs to keep a wary eye on the code quality of projects developed for this area. The PVS-Studio static analyzer is one of the solutions to this problem. Today, the Apache Flink project developed by the Apache Software Foundation, one of the leaders in the Big Data software market, was chosen as a test subject for the analyzer.

So, what is Apache Flink? It is an open-source framework for distributed processing of large amounts of data. It was developed as an alternative to Hadoop MapReduce in 2010 at the Technical University of Berlin. The framework is based on the distributed execution engine for batch and streaming data processing applications. This engine is written in Java and Scala. Today, Apache Flink can be used in projects written using Java, Scala, Python, and even SQL.

Source de l’article sur DZONE

/par

Display the Latest News on Your Site With Mediastack

,

What an extraordinary year 2020 has been for the news! From the ongoing coronavirus crisis, to a turbulent US election, to the unrelenting march of Bitcoin, this year like no other we’ve been glued to our phones micro-analyzing every tidbit of news.

Which makes this the perfect time for mediastack, an awesome REST API that allows you to embed a customizable news feed, sourced from the world’s top news agencies, and updated by the minute, right on your site.

Integrating Global News with Your Site

News is the beating pulse of so many global industries. From political decisions that affect stock prices, to natural disasters that interrupt goods and services, to the whims of celebrities who overnight transform brands from unknown to must-have.

Whether you’re building a site for a non-profit in Louisiana that cares deeply about both Washington politics, and hurricanes in the Caribbean; or you’re building an app for a golf course in Halkidiki that’s focused on both local news, and golf around the world; delivering real-time news content to those users elevates UX.

Tightly integrating the news with your site makes it a hub for users hungry for that very news. The only limit is your creativity.

Display Up-to-Date News on Your Site

When news breaks around the world the top networks scramble to catch up; they simply can’t maintain correspondents in every town and city in the world, and so they rely on affiliates. mediastack pulls in news from over 7,500 different sources in over 50 countries worldwide, giving you access to exactly the same affiliates frequently used by big news organizations like CNN, MSNBC, BBC, or ABC.

When it’s one of the big players in news that breaks a story first, mediastack still has you covered because as will as covering smaller, lesser-known sources mediastack delivers real-time news from CBS, Sky News, The Guardian, Al Jazeera, USA Today, and a host of trusted names across the industry.

If your site targets users that are only interested in certain types of story — like sports, or Hollywood celebrities — then you can even pull in stories from ESPN, TMZ, or Fox News.

Get Started Quickly with mediastack

Getting started with mediastack couldn’t be simpler, and there’s a free plan that’s more than enough to prototype your project.

Full documentation is provided with code examples for PHP, Python, jQuery, Go, and Ruby. To start integrating all you need to do is register for a free access key.

Once you have your free access key, you connect to the API, then customize the results you receive with simple parameters. You can specify the types of news, the precise sources (including omitting sources), languages, countries, and most importantly your keywords.

For example here’s how you’d request science news from CNN, but not TMZ:

https://api.mediastack.com/v1/news
?access_key=[ INSERT YOUR ACCESS KEY HERE ]
&categories=science
&sources=cnn,-tmz

Let’s say you want to display Spanish language crypto news on your site, it couldn’t be easier:

https://api.mediastack.com/v1/news
?access_key=[ INSERT YOUR ACCESS KEY HERE ]
&categories=business,technology
&languages=es
&search=crypto,bitcoin,btc,xrp,ripple,etherium,altcoin

The API sends back simple JSON data that’s easy to run through. Each news item includes the author, title, description, url, source, image, category, language, country, and a published_at timestamp that records when the story was posted.

Once the feed is setup, sit back and relax. It’s all automated from now on.

The Best Source of News for Your Website

mediastack is delivered by apilayer, quite rightly one of the most trusted names in APIs, and is capable of handling millions of requests simultaneously.

Fast, updated by the minute, highly customizable, reliable, and sourced from the biggest names in the news industry, mediastack is an amazing API.

There’s a free-forever plan that allows you to use the API without charge, for up to 500 API calls per month, that’s perfect for trying it out.

For commercial use, plans start at just $19.99/month, and can handle up to 250,000 calls per month. Commercial plans also include HTTPS encryption, live news delivery, access to historical data, and — should you ever need it — technical support.

Head over to mediastack today, to prepare your site for whatever events 2021 throws at us.

 

[– This is a sponsored post on behalf of mediastack –]

Source


Source de l’article sur Webdesignerdepot

/par

5 NLP Trends to Watch in 2021

,

Although still in its infancy, 2020 has been a year of significant growth for Natural Language Processing (NLP). In fact, research from Gradient Flow found that even in the wake of the COVID-19 pandemic, 53% of technical leaders indicated their NLP budget was at least 10% higher compared to 2019, with 31% stating their budget was at least 30% higher than the previous year. This is quite significant, given most companies are experiencing a downturn in IT budgets, as companies adjusted their spending in response to the pandemic. 

With the power to help streamline and even automate tasks across industries, from finance and healthcare to retail and sales, leaders are just beginning to reap the benefits of NLP. As the technology advances further and its value becomes more widely known, NLP can achieve outcomes from handling customer service queries to more mission-critical tasks, like detecting and preventing adverse drug events in a clinical setting. As NLP continues on its growth trajectory, here are some of the top trends to watch in 2021. 

Source de l’article sur DZONE

/par

7 Essential Tools for a Competent Data Scientist

,

A data scientist extracts manipulate and generate insights from humongous data. To leverage the power of data science, data scientists apply statistics, programming languages, data visualization, databases, etc.

So, when we observe the required skills for a data scientist in any job description, we understand that data science is mainly associated with Python, SQL, and R. The common skills and knowledge expected from a data scientist in the data science industry includes – Probability, Statistics, Calculus, Algebra, Programming, data visualization, machine learning, deep learning, and cloud computing. Also, they expect non-technical skills like business acumen, communication, and intellectual curiosity.

Source de l’article sur DZONE

/par

Editors’ Picks: Summer 2020

,

Agile 

AI

Big Data

Cloud

Database

DevOps

Integration

  • Mulesoft 4: Continuous Delivery/Deployment With Maven by Ashok S — This article is a great example of what we want every tutorial to look like on DZone. The main aim of this article is to provide a standard mechanism to release project artifacts and deploy to Anypoint Platform, from the local machine or configure in continuous delivery pipelines.
  • Integration With Social Media Platforms Series (Part 1) by Sravan Lingam — This article helps you to build a RESTful API through MuleSoft that integrates with LinkedIn and shares a post on behalf of one’s personal account. I like this article because, in the age of social media, it’s so important for businesses to be connected and integrated!

IoT

Java

Microservices

Open Source

Performance

  • What Is Big O Notation? by Huyen Pham — Aside from a silly name, this article is an example of an in-depth analysis on a little-spoken-about concept. In this article, take a look at a short guide to get to know Big O Notation and its usages.
  • Is Python the Future of Programming? by Shormisthsa Chatterjee — Where is programming going? This article attempts to answer this question in a well-rounded way. The author writes, "Python will be the language of the future. Testers will have to upgrade their skills and learn these languages to tame the AI and ML tools".

Security

Web Dev

  • A Better Way to Learn Python by Manas Dash: There’s so many resources available for learning Python — so many that it’s difficult to find a good and flexible place to start. Check out Manas’ curated list of courses, articles, projects, etc. to get your Python journey started today. 
  • Discovering Rust by Joaquin Caro: I’m a sucker for good Rust content, as there’s still so many gaps in what’s available. Joaquin does a great job of giving readers his perspective of the language’s features in a way that traditional docs just 

Source de l’article sur DZONE

/par

3 Ways to Select Features Using Machine Learning Algorithms in Python

,

Artificial intelligence which gives machines the ability to think and behave like humans are gaining traction since the last decade. These features of artificial intelligence are only there because of its ability to predict certain things accurately, these predictions are based upon one certain technology which we know as machine learning (ML). Machine learning as the name suggests is the computer’s ability to learn new things and improve its functionality over time. The main focus of machine learning is on the development of computer programs that are capable of accessing data and using it to learn for themselves. 

To implement machine learning algorithms, two programming languages, R and Python for machine learning are normally used. Generally, selecting features for training data on machine learning in python is a very complex and technical process. But here we will go over some basic techniques and details regarding what is machine learning and how it works. So, let us start by going into detail regarding what ML is, what feature selection is and how can one select feature using python.

Source de l’article sur DZONE

/par