Hadoop User Group (HUG) Ireland and the Return of the Big Data Jedi

The 12th’s return by HUG Ireland in association with Oracle... A September Special for all Big Data lovers…

Whilst the team at HUG Ireland took an event moratorium in August, we have not wasted a minute in setting up a cracking Autumn 2016 schedule for our members, which forms Dublin’s premiere technical Big Data community. November and October already have confirmed speakers covering areas of Data Science and Data Pipelines that should not be missed by one and all!

We have also been busy with the team at Oracle to bring you a great lineout for September 12th at the Morrison Hotel on Ormond Quay from 6pm. Two quality speakers are speaking on the evening on The Future of Analytics, Oracle Big Data SQL and The Three I"s of Big Data with a bar on premises, so we can all catch up with each other on site on what’s new; before and after the quality presentations by Mark Rittman, CTO of Rittman Mead and Ian Sharp, Director of Data Science with Oracle UK.

So, some background on our speakers:

Mark is CTO and co-founder at Rittman Mead, an Oracle (and more recently, Cloudera) Partner specialising in BI, big data and data warehousing implementations, training and support. Mark is an Oracle ACE Director, author of two Oracle Press books and leads Rittman Mead's customer projects around Hadoop, cloud and machine learning.

Ian is a Data Scientist at Oracle with over 15 experience helping customers deliver insight and innovation around business intelligence, Data Warehousing and Big Data. He has worked across industries with organisations such as Worldpay, Sky, NHS, BT, HSBC, AVIVA, the NHS, Bord Gais and the Discovery Channel.

The agenda for the evening from 6pm looks like this:

The Future of Analytics, Data Integration and Data Driven Applications on Big Data Platforms by Mark Rittman, CTO of Rittman Mead

Up until now thought of as a cheap storage offloading layer for more capable relational data warehouses and BI tools, Hadoop is fast maturing into the default platform for all analytic and data integration workloads. In this session we'll look at how technologies such as Apache Arrow, Apache Kudu and Apache Drill will bring in-memory, column-store BI functionality to Hadoop but extend that to cover flexible-schema data held in data lakes and NoSQL databases; we'll see how Apache Spark, Apache Kafka and real-time streaming functionality coming to vendor ETL tools such as Oracle Data Integrator will enable real-time loading and integration with traditional relational databases; and finally we'll see how a new generation of e-commerce, logistics and customer applications will take advantage of the machine learning and predictive analytics capabilities provided by Hadoop to add smarts and intelligence to day-to-day operational applications.

Using Oracle Big Data SQL to add Hadoop + NoSQL to your Oracle Data warehouse by Mark Rittman, CTO of Rittman Mead

Oracle Big Data SQL 3.0 extends the Oracle SQL dialect, security framework and advanced analytics capabilities of the Oracle database to big data platforms, initially just for Oracle Big Data Appliance but now made available for commodity Cloudera and Hortonworks clusters. Complementing basic Apache Hive but competing with other next-generation SQL-on-Hadoop solutions such as Impala, Presto and Drill, Oracle Big Data SQL lets you work with big data sources as if they were another set of external tables – and can bring Exadata style query offloading to this significantly lower-cost platform. In this session we’ll explore at a technical level how Big Data SQL works (including the key limitations and “gotchas”), see how features such as storage indexes and storage handlers introduced with the 2.0 release significantly improve what the product can do, how statistical and advanced analytics features in Oracle Database Enterprise Edition can also be used on Hadoop sources, understand at a high-level what bringing Oracle security Hadoop means – and see if it’s really now possible to use Big Data SQL without the previous Exadata and BDA prerequisites.  

The Three I"s of Big Data, Insight, Innovation and Identify by Ian Sharp,  Lead Data Scientist with Oracle UK

Insight and innovation aside, Big Data Social Media is increasingly about Identity; who we are, who we aspire to be, and in the case of Fraud and Theft, who we pretend to be. Data Science, Big Data Analytics and Machine Learning have emerged as the prime technologies and disciplines to distill nuances and value around these identities. This presentation discusses the challenges and opportunities of big data analytics against polyglot sources, the evolution of new languages and frameworks such as R, Python, SciKit Learn and Zeppelin and the operational settings  around wider BI consumption, Streams and Apps APIs.  Using leading edge reference cases, we will explore  the stories and focus on the challenges for integration, governance and stability faced  in their  transition from Hadoop driven Innovation  to Mainstream business success. Finally we will highlight  some up and coming trends for the industry to watch

So, RSVP today and check out our twitter hashtag at #HUGIreland for the latest from our community.  Looking forward to seeing you all on the 12th!

Sources/Credits:

Pics:

Hadoop User Group (HUG) Ireland

Credits:

Morrison Location:

https://www.google.ie/maps/place/The+Morrison,+a+DoubleTree+by+Hilton+Hotel/@53.3466752,-6.2681787,17z/data=!3m1!4b1!4m5!3m4!1s0x48670e83401492d5:0xdc556b1fc3ff0b3b!8m2!3d53.346672!4d-6.26599

Event Page:

http://www.meetup.com/hadoop-user-group-ireland/

Speaker Profiles:

Ian:

https://www.linkedin.com/in/ian-sharp-03a3835?authType=NAME_SEARCH&authToken=-usr&locale=en_US&trk=tyah&trkInfo=clickedVertical%3Amynetwork%2CclickedEntityId%3A15847534%2CauthType%3ANAME_SEARCH%2Cidx%3A1-3-3%2CtarId%3A1471963329454%2Ctas%3Aian%20sharp

https://www.oracle.com/uk/index.html

Mark:

https://www.linkedin.com/in/markrittman?authType=NAME_SEARCH&authToken=3wfo&locale=en_US&srchid=2114422741471963406937&srchindex=1&srchtotal=3&trk=vsrp_people_res_name&trkInfo=VSRPsearchId%3A2114422741471963406937%2CVSRPtargetId%3A2087871%2CVSRPcmpt%3Aprimary%2CVSRPnm%3Atrue%2CauthType%3ANAME_SEARCH

http://www.rittmanmead.com/

Python as a Software Language - Does Sexy ever go out of Style?

Python’s lineage in science defines its past, but does it define future direction? Is Python 3.4 here to stay?

There has being allot of talk coming from certain quarters about Python being a dying language, which of course is not true. Python’s adoption is increasing and since finishing my last course in Software and Cloud Technology, I have heard that Python technology, community and jobs becoming more plentiful, which is great news indeed (...if you like groovy software!).

All computer science or software students will tell you that Python is a dynamic language that is learnt after learning strongly typed languages like C# or Java. For instance, I learned Object Orientated Programming (OOP) through C# at college and a little over 6 months later am programming in Java and Python alongside other languages at entry level. This “newbie” first impression of Python (compared let’s say to C# and Java, which are similar to each other) has led me to to consider it a sexy language for the following reasons:

a.              Syntax is concise, you literally can do more with less. You don’t have to programmatically write a novel to make a programmatically correct point!

b.              Don’t have to worry about excessive {wrapping of methods} making methods more readable

c.              Function nesting is actually fun

d.              Dynamic typing of data types allows you to move through variable declarations much easier and variable typing is smooth allowing good data typing to take place dynamically. E.g. int x = 12; in C#/Java is x = 12 in Python.

e.              Mathematical and Scientific libraries are awesome which is a clear advantage over general purpose languages like Java or C#. Am only scratching the surface of libraries like numpy and scipy but intend with time to get more immersed in them.

Downside for me compared to Java and C# is the following:

1. Interpreter language so you need to run it to see a programming mistake. Don’t run your program regularly and use good development techniques, you could be forever debugging your code. Compiler warns you when typing of certain problems in Java and C#
2. For those like myself who are not gurus yet in the language, we code heavy using the libraries and intelisense. Better IDEs for little to no cost to compete with Java and C# would bring more new (and poor) programmers like myself to the language.
3. Python’s original use case is scientific but it’s very good for example in web development. Wider adoption due to its “science/numbers” reputation may be slower in my view than it should be.
4. Python 2.7 (established and awesome) V 3.4 (why are we all not using it?). Transitioning to Python 3 is a very long process with 2020 as the cut off date for supporting Python 2 according to Thomas Robitaille’s article reporting on his Python adoption survey.

So, on the primary use of Python 3, Thomas’s survey 2012 (313 respondents) saw Python 3 uptake at 0% rise to 17% in 2015 (781 validated respondents) for primary use, which is a note that its slowly rising in adoption. There is a myriad of reasons why it is such a long process and a slow rising number on Python 3. Some of those reasons are as follows:

a.        Scientific libraries are fully compatible with Python 3 raising that number from 0%.
b.              Some use-cases are apparently not supported yet in Python 3
c.              Python 2 is still being used in training for the language (online & classroom)
d.              Company migration is not as fast as it could be from Python 2 to Python 3 restricting what version developers use
e.              Windows use of Python (originally on Linux/Mac) is up-to 10% thanks to continuous integration testing from developments such as AppVeyor giving Python further penetration power into Enterprise.

Here’s the kicker, when an open source language has two stable versions that are popular and is strongly supported by the community, it delivers a ‘slow burn’ transition possibility to Version 3 by 2020, when it just may outclass all others in areas that would not normally consider Python as an option. Even now, Python’s use cases has expanded from its core scientific beginnings to do things like web development to calculating the distance between stars in other galaxies, so it’s not hard to why one may term it as sexy language. Why not leave your opinion. Is Python a language worth learning?

Sources/Credits:

Pics;

www.linkedIn.com

www.arpaso.com

www.pyfound.blogspot.com

www.pytennassee.tumblr.com

Credits;

https://en.wikipedia.org/wiki/Python_(programming_language)

https://pythonconquerstheuniverse.wordpress.com/2009/10/03/static-vs-dynamic-typing-of-programming-languages/

http://searchwindevelopment.techtarget.com/definition/C

https://realpython.com/blog/python/inner-functions-what-are-they-good-for/

http://astrofrog.github.io/blog/2015/05/09/2015-survey-results/