scala - SQL queries in RDD -


i must operate rdd scala/spark methods , sql queries.

is possible operate rdd directly via sql queries?

the proposed ways (schemardd or dataframe) require memory leakage.

after such transformation have in memory 2 identical huge objects.

yes, in way, may able so. you'll need create own version of dataframe.

dataframe abstraction on rdds. nevertheless, joins, filters, etc. features find spark-sql optimized dataframes made on rdds first.


Comments

Post a Comment

Popular posts from this blog

android - net_scheduler holding wakelock -

i have observed on occasions, net_scheduler keeps acquiring wakelock repeatedly on short durations , prevents android device sleep. from logcat, not able obtain conclusive information why might happening. of gms services active. can suggest when there higher chance occur perspective of gms services. this seems coming gcmnetworkmanager , schedules network-oriented tasks, google play services able batch network operations across system. also if use more advanced scheduling firebase jobdispatcher , internally gcmnetworkmanager used.
Read more

sql - MySQL : Getting Entries from a many-to-many table -

i have following table, select records should have mentioned values. example select companyid should have 3 providerid 200,300,400. running query should retrun companyid 2. opposite of in . in advance ! companyid | providerid -------------------------------------- 1 100 2 200 3 500 4 600 2 300 2 400 7 100 do group by , select rows of 3 wanted employeeid values. use having make sure 3 different employeeid values. select companyid tablename employeeid in (200, 300, 400) group companyid having count(distinct employeeid) = 3 another approach double self join, 1 each employeeid. however, less flexible: select distinct t1.companyid (select companyid tablename employeeid = 200) t1 join (select companyid tablename employeeid = 300) t2 on t1.companyid = t2.companyid join (select companyid tablename employeeid = 400)...
Read more

java - Retrieving data from database using jsp (Hibernate + Spring + Maven) -

hello, guys. i'm trying retrieve data oracle database, nothing appears on screen. here userslist.jsp <%@page contenttype="text/html" pageencoding="utf-8"%> <!doctype html> <html> <head> <title>spring mvc hello world</title> </head> <body> <h2>all users in system</h2> <table border="1"> <tr> <th>user id</th> <th>login</th> </tr> <c:foreach items="${users}" var="user"> <tr> <td>${user.userid}</td> <td>${user.login}</td> </tr> </c:foreach> </table> </body> </html> it seems ok. , here spring controller userscontroller.java package com.controller; import com.database.users; import com.service.usersmanager; import java.util.list; impo...
Read more