I have 2 Python modules/scripts - task.py and runner.py. First one
(task.py) is a little Spark job and works perfectly well by itself.
However, when called from runner.py with exactly the same arguments, it
fails with only useless message (both - in terminal and worker logs).
org.apache.spark.SparkException: Python worker exited unexpectedly
(crashed)
Below there's code for both - task
Folks,
I wrote the following wrapper on top on combineByKey. The RDD is of
Array[Any] and I am extracting a field at a given index for combining.
There are two ways in which I tried this:
Option A: leave colIndex abstract in Aggregator class and define in derived
object Aggtor with value -1. It is set later in function myAggregate. Works
fine but I want to keep the API user unaware of colIndex
Hi, I want to use pySpark with yarn. But documentation doesn't give me full
understanding on what's going on, and I simply don't understand code. So:
1) How python shipped to cluster? Should machines in cluster already have
python?
2) What happens when I write some python code in "map" function - is it
shipped to cluster and just executed on it? How it understand all
dependencies, which my
Hi,
I have a query that I'm attempting to load from a PHP page that is not working. However, the exact same query works just fine when I run it from the MySQL command line. This is the query:LOAD DATA LOCAL INFILE '/home/ratdog/public_html/test.txt' INTO TABLE `shows` I've run similar queries through PHP with no problem, and I'm absolutely stumped on why this won't work when it works fine from the
Hi Everyone.
I had a perfectly good working version of MySQL and then I decided to
try something new... Now, I can start up as root with mysqld, but
when I start with mysqld_safe as I always have in the past, mysqld
quits immediately. Here's the saga:
I decided to use phpmyadmin to change all my user passwords to SHA1.
After reloading MySQL, I couldn't connect with ANY password.
Hi!
I have had MySQL up and running for some time on my Linux box. Everything
has been working wonderfully till one of my programs "died". I can no longer
do anything to the MySQL server!
When I now start mysqld 4 processes start. If I then type mysqlshow at the
command, nothing happens, and I get no answer. I cannot telnet to port 3306
either. No other programs seem to work.
I stopped
Ok, I finally got replication to work across two Win2k servers. But now
here is my dilemma. Whenever the service is stopped on the slave, any
tables that are updated on the master become corrupt on the slave when it
starts up again. Is this normal? Maybe I'm missing something but that
doesn't seem like a good plan. I read in the manual that it is okay if the
master shuts down (which I haven
Hi all, i am sending a http request from jmeter-2.9 and jmeter-2.11.
Jmeter-2.11 shows exception in response for alternate requests. The script
works well in jmeter-2.9 without any issues.
I placed a breakpoint in my server to see whether the alternate requests
are reaching my server. But its not reaching the server at all. My scripts
directly points to the server.
The response error that
Hi all,
My code was working fine in spark 1.0.2 ,but after upgrading to 1.1.0, its throwing exceptions and tasks are getting failed.
The code contains some map and filter transformations followed by groupByKey (reduceByKey in another code ). What I could find out is that the code works fine until groupByKey or reduceByKey in both versions.But after that the following errors show up in Spark 1
Hi all,
I have several Hive queries that work in spark-shell, but they don't work in
spark-submit. In fact, I can't even show all databases. The following works
in spark-shell:
import org.apache.spark._
import org.apache.spark.sql._
object ViewabilityFetchInsertDailyHive {
def main() {
val x = sqlContext.sql("show databases")
val z = x.collect
for(i
Hi,
The search doesn't work on the website interface here:
http://208.64.71.46:8080/en/
but it works when I run 'bin/*nutch*
org.apache.*nutch*.searcher.*NutchBean*keyword' on shell, I got lots
of search results. I wonder what could be
wrong?
Thanks!
Tony
--
Signature: Success is a journey that never ends.
Hi there,
I have a strange situation on one of my tables in HBase:
* disable/describe 'my_table' works (using HBase shell)
* truncate/drop doesn't - yells table does not exists.
How do I fix it?
HBase Shell quote:
hbase(main):003:0> disable 'my_table'
0 row(s) in 0.0480 seconds
hbase(main):004:0> truncate 'my_table'
Truncating 'my_table' table (it may take a while):
ERROR: Unknown table my_table
Hello everybody
I saw the unusual status on WEB UI(see the attached image). But HBase
Shell still works fine. I can scan, put, create table, drop table.
Sometimes restarting HBase can get rid of this status, but this time not
work.
How can I resolve this error? I've tried "hbase hbck -fix", "hbase hbck
-fisAssignments", "hbase hbck -repair".......The status is still the same.
When looking
Hi,
We have a custom jar that when used in Beeswax makes job fail but works from hive shell.
Error is something like:
================
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row [Error getting row data with exception java.lang.ClassCastException: java.util.ArrayList cannot be cast to com.groupon.hive.serde.TDFastExportRowObject
at
Hi to all here is my problem...
executing this in the shell i got this:
>db.users.find({name : /^Lu/})
{ "_id" : ObjectId("4d1126760e760b1996acb404"), "name" : "Luigi 1",
"birthdate" : NumberLong("1292969548867"), "city" : { "name" : "Rome",
"code" : "RM" } }
{ "_id" : ObjectId("4d1127926de00b19987fa819"), "name" : "Luigi 2",
"birthdate" : NumberLong("1292969874625"), "city" : { "name" :
"
https://github.com/neo4j/neo4j/issues/1789
Is there any update to this issue?
When it might be fixed?
Thanks!
Hi,
I have set up a cluster with Mesos (backed by Zookeeper) with three
master and three slave instances. I set up Spark (git HEAD) for use
with Mesos according to this manual:
http://people.apache.org/~pwendell/catalyst-docs/running-on-mesos.html
Using the spark-shell, I can connect to this cluster and do simple RDD
operations, but the same code in a Scala class and executed via sbt
run
Hello everyone,
I have some client code running on a PC that
interfaces with a Java server on a different PC.
There is a connection made via the use of Sockets, and
with the aid of Helpers I query a database and return
the results to a simple Console/DOS window
application.
I am getting unstuck when I attempt to implement the
Client side socket creation inside an applet.
//-- This
using 99999 as the DATE_ADD interval value will result in 000-00-00 but
root# mysql --version
mysql Ver 14.12 Distrib 5.0.41, for pc-linux-gnu (i686) using EditLine
wrapper
CREATE TABLE `Users` (
`CoreID` int(10) unsigned NOT NULL auto_increment,
`Username` varchar(155) default NULL,
`Password` varchar(64) default NULL,
`password_expire` timestamp NOT NULL default '0000-00-00
00
Get rid of the offending $hostname.index file that keeps track of
binlogs (Thanks, Olaf!).
2. Fix permissions (Thanks, Gerald Clark!).
Have a nice day!
Chris
