Almost there

What i did today

Changes -> [commit]

A function which wasn't called (and it also isn't necessary anymore since there's a Gst.PadProbeInfo.get_buffer() in the new GStreamer) :
https://github.com/bananadine/flumotion/commit/51cd0e0566fa0c268320b1772d4c96a4664acd96#L0R785

I reverted some of the changes i did to the probes:
https://github.com/bananadine/flumotion/commit/51cd0e0566fa0c268320b1772d4c96a4664acd96#L0R1023
https://github.com/bananadine/flumotion/commit/51cd0e0566fa0c268320b1772d4c96a4664acd96#L0R1037

Ported some missed code:
https://github.com/bananadine/flumotion/commit/51cd0e0566fa0c268320b1772d4c96a4664acd96#L0L943

The outcome

Worker output -> http://pastebin.com/Rnxx1jMt

Manager output -> http://pastebin.com/ep8S1yvg

TO DO:
See why this happens:

1. (flumotion-job:20335): GStreamer-CRITICAL **: gst_mini_object_ref: assertion `mini_object != NULL' failed
2. WARN  [20328]                                  component-job-heaven Aug 01 18:54:30      Child with pid 20335 segfaulted. (flumotion/common/worker.py:85)
3. INFO  [20328]                                  component-job-heaven Aug 01 18:54:30      Core dumped. (flumotion/common/worker.py:105)
4. WARN  [20328]                                  worker            Aug 01 18:57:24      Lost connection to manager, will attempt to reconnect (flumotion/worker/medium.py:79)
5. INFO  [20328]                                  workerbrain       Aug 01 18:57:24      Reactor shutting down, stopping jobHeaven (flumotion/worker/worker.py:190)
6. INFO  [20328]                                  worker            Aug 01 18:57:24      Stopping worker 'localhost' (flumotion/common/process.py:58)
7. INFO  [20328]                                  component-job-heaven Aug 01 18:57:24      Reaped child with pid 20332 signaled by signal 1. (flumotion/common/worker.py:93)
8. WARN  [20328]                                  component-job-heaven Aug 01 18:57:24      No core dump generated. Were core dumps enabled at the start ? (flumotion/common/worker.py:96)
9. WARN  [20328]                                  workermedium      Aug 01 18:57:24      Tried to callRemote(componentAddMessage), but we are disconnected (flumotion/common/medium.py:121)

And get the stream working

EDIT: Tomorrow i have to travel (School related) and will be back Saturday night or Sunday morning.

TypeError: unknown type GstIntRange "FIXED"

What i did

Until i apply a patch for gst-python, the line where it gave me 'TypeError: unknown type GstIntRange' will be like this -> [commit]

There was an error i got for 'self.pipeline.set_start_time(Gst.CLOCK_TIME_NONE)' sayng -1 (the value of GST_CLOCK_TIME_NONE) is out of range, but the equivalent to that (2**64-1) works. I reported the bug to introspection. [commit]

Other changes i did, here -> [commit]

What needs to be done

Here's the current worker output -> http://pastebin.com/Du4wbviZ

Everything works great until the muxer shows up, then i get an internal flow error. My guess is that this is due to the probes.
In GStreamer 1.0 add_probe and remove_probe replace a lot of functions from GStreamer 0.10 and i had to save the return values of add_probe to use them for removing the probes. The code i wrote is messy and i'm sure it contains a lot of mistakes, but that's because i get confused by all those probes.

TO DO:

Fix the internal flow error, get a working stream.
I don't think i can fix this on my own, i'll make a new post if i get this working.

Progress

The video test producer's caps could not be parsed because of the formatting. I fixed that with correct placement of commas -> [7c12ca0]

Other changes -> [5a1ac72]

Checklist:

1.

1. Traceback (most recent call last):
2.   File "/home/max/Desktop/flumotion-porting/flumotion/flumotion/common/gstreamer.py", line 51, in verbose_deep_notify_cb
3.     value = orig.get_property(pspec.name)
4. AttributeError: 'NoneType' object has no attribute 'name'

Printing the arguments in verbose_deep_notify_cb in gstreamer.py gives the following:

'print object' prints <Pipeline object at 0x9cb0c34 (GstPipeline at 0x9ce6c38)>

'print orig' prints <__main__.GstMultiFdSink object at 0x9868c0c (GstMultiFdSink at 0x98656d0)>

'print pspec' prints None

'print component' prints all components in the pipeline

Printing pspec in the pre-ported Flumotion shows that it's only a log of the pipeline properties/specifications.

Tried a lot of things, but at the end i did this to get pass the problem:

https://github.com/bananadine/flumotion/commit/5a1ac72140408aeb84be09cc99de792c01390fcd#L0R51

2.

1. Twisted traceback:
2. Traceback (most recent call last):
3.   File "/usr/lib/python2.7/dist-packages/twisted/spread/pb.py", line 896, in proto_message
4.     self._recvMessage(self.localObjectForID, requestID, objectID, message, answerRequired, netArgs, netKw)
5.   File "/usr/lib/python2.7/dist-packages/twisted/spread/pb.py", line 910, in _recvMessage
6.     netResult = object.remoteMessageReceived(self, message, netArgs, netKw)
7.   File "/home/max/Desktop/flumotion-porting/flumotion/flumotion/common/medium.py", line 279, in remoteMessageReceived
8.     self, broker, message, args, kw)
9.   File "/home/max/Desktop/flumotion-porting/flumotion/flumotion/twisted/pb.py", line 584, in remoteMessageReceived
10.     d = defer.maybeDeferred(method, *args, **kwargs)
11. --- <exception caught here> ---
12.   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 134, in maybeDeferred
13.     result = f(*args, **kw)
14.   File "/home/max/Desktop/flumotion-porting/flumotion/cache/component/3a795ff0d790d96af4e17e4306ad0c5c/flumotion/component/feedcomponent.py", line 249, in remote_setMasterClock
15.     return self.comp.set_master_clock(ip, port, base_time)
16.   File "/home/max/Desktop/flumotion-porting/flumotion/cache/component/3a795ff0d790d96af4e17e4306ad0c5c/flumotion/component/feedcomponent010.py", line 504, in set_master_clock
17.     element = self.pipeline.get_by_name('src')
18. exceptions.TypeError: unbound method get_by_name() must be called with Pipeline instance as first argument (got str instance instead)

Got rid of this:

https://github.com/bananadine/flumotion/commit/5a1ac72140408aeb84be09cc99de792c01390fcd#L0R51

3. 

1. Traceback (most recent call last):
2.   File "/home/max/Desktop/flumotion-porting/flumotion/cache/vorbis-component/2b18938d80c0f3474d96f6eea8f6c471/flumotion/component/encoders/vorbis/vorbis010.py", line 75, in buffer_probe
3.     in_rate = kaps[0].get_value('rate')
4.   File "/usr/lib/python2.7/dist-packages/gi/types.py", line 43, in function
5.     return info.invoke(*args, **kwargs)
6. TypeError: unknown type GstIntRange

(pygst doesn't yet support the full set of gstreamer GType values)

4.

1. WARN  [13016] "encoder-audio"                  feedcomponent     Jul 28 16:42:14      element /GstPipeline:pipeline-encoder-audio/GstFdSrc:eater:default error Internal data flow error. gstbasesrc.c(2809): gst_base_src_loop (): /GstPipeline:pipeline-encoder-audio/GstFdSrc:eater:default:
2. streaming task paused, reason not-negotiated (-4) (flumotion/component/feedcomponent010.py:261)
3. INFO  [13016] "encoder-audio"                  feedcomponent     Jul 28 16:42:14      We got an eos from pipeline-encoder-audio (flumotion/component/feedcomponent010.py:284)

I don't know anything about this one, yet.

5.

1. Twisted traceback:
2. Traceback (most recent call last):
3.   File "/usr/lib/python2.7/dist-packages/twisted/python/log.py", line 84, in callWithLogger
4.     return callWithContext({"system": lp}, func, *args, **kw)
5.   File "/usr/lib/python2.7/dist-packages/twisted/python/log.py", line 69, in callWithContext
6.     return context.call({ILogContext: newCtx}, func, *args, **kw)
7.   File "/usr/lib/python2.7/dist-packages/twisted/python/context.py", line 118, in callWithContext
8.     return self.currentContext().callWithContext(ctx, func, *args, **kw)
9.   File "/usr/lib/python2.7/dist-packages/twisted/python/context.py", line 81, in callWithContext
10.     return func(*args,**kw)
11. --- <exception caught here> ---
12.   File "/usr/lib/python2.7/dist-packages/twisted/internet/posixbase.py", line 586, in _doReadOrWrite
13.     why = selectable.doRead()
14.   File "/home/max/Desktop/flumotion-porting/flumotion/flumotion/twisted/fdserver.py", line 98, in doRead
15.     message[offset:offset+msglen])
16.   File "/home/max/Desktop/flumotion-porting/flumotion/flumotion/job/job.py", line 338, in fileDescriptorsReceived
17.     os.close, eaterId)
18.   File "/home/max/Desktop/flumotion-porting/flumotion/cache/component/3a795ff0d790d96af4e17e4306ad0c5c/flumotion/component/feedcomponent010.py", line 806, in feedToFD
19.     self.pipeline.get_state(0)[1] == Gst.State.NULL):
20. exceptions.TypeError: unbound method get_state() must be called with Pipeline instance as first argument (got int instance instead)

This is gone.

6.

1. Traceback (most recent call last):
2.   File "/home/max/Desktop/flumotion-porting/flumotion/cache/component/3a795ff0d790d96af4e17e4306ad0c5c/flumotion/component/feedcomponent.py", line 985, in buffer_probe_cb
3.     pad = Gst.Element.get_static_pad('src')
4. TypeError: unbound method get_static_pad() must be called with Element instance as first argument (got str instance instead)

This change needs to be applied here: https://github.com/bananadine/flumotion/commit/5a1ac72140408aeb84be09cc99de792c01390fcd#L1L1019

Instead of calling the method on Gst.Element (which makes no sense) I need to call it on something which is a GstElement.

I'll post the current worker output tomorrow/later in an edit to show what needs to be fixed.

EDIT: Worker output http://pastebin.com/xsa7x7rj

Some things i did today

The callback functions need a lot of work. Issues number 1, 5 and 6 are callback related.
Here are some changes i did today, most of the time i spent analyzing and inspecting:
https://github.com/bananadine/flumotion/commit/08fd2a9afa5d1a017f1bb5427c641b91413530f5

Here's what I've found out about the issues mentioned in the previous post

1.
'verbose_deep_notify_cb' is a function in gstreamer.py and it's called in '_setup_pipeline' in feedcomponent010.py  by 'self.pipeline.connect' . I couldn't find any information on 'connect' and there is no such function in Gst.Pipeline, but there is a Gst.Element.connect()

2.
I tried printing out self.pipeline_string after this line:
https://github.com/bananadine/flumotion/blob/master/flumotion/component/feedcomponent.py#L337
I got multiple print outs, here they are in the worker log along with everything else from the log: http://pastebin.com/b5XU43xA
I'd like to take out one of the print outs which i think is important:

1. videotestsrc is-live=true name=source ! identity name=identity silent=TRUE ! None ! gdppay name=feeder:default-pay ! multifdsink sync=false name=feeder:default buffers-max=500 buffers-soft-max=450 recover-policy=1

There is a 'None' only in this one and it's for the video test producer. I need to find out where and what gets a None value and fix that.

3.
self.pipeline is initially given a None value and it seems that doesn't change, i need to figure out why is that. Anyway, regarding 'set_new_stream_time':
Gst.Pipeline no longer has a new_stream_time(), Gst.Element has set_start_time() instead.
Also, Gst.Element has the function set_base_time()

5.  Fixed how many arguments are givent o event_probe_cb(). Now i get this:

1. Traceback (most recent call last):
2.   File "/home/max/Desktop/flumotion-porting/flumotion/cache/component/b749e957772674b6a3071482d5a39e08/flumotion/component/feedcomponent.py", line 1019, in event_probe_cb
3.     if 'audio' not in caps[0].to_string():
4. TypeError: 'Caps' object does not support indexing

Need to fix this tomorrow.




6. Same as above, but i get this instead:

1. Traceback (most recent call last):
2.   File "/home/max/Desktop/flumotion-porting/flumotion/cache/component/b749e957772674b6a3071482d5a39e08/flumotion/component/feedcomponent.py", line 986, in buffer_probe_cb
3.     pad = Gst.Element.get_static_pad('src')
4. TypeError: unbound method get_static_pad() must be called with Element instance as first argument (got str instance instead)

The old function used to work with strings now i need to give 'get_static_pad' an Element instance.

Replaced TIME_ARGS and some other stuff

What i did today

Here are the commits:
https://github.com/bananadine/flumotion/commit/08ef0fad603099e3d3dbc8d6ebd9bad568e88e46
https://github.com/bananadine/flumotion/commit/9f3b804afbe7ed3970919b3679a872f83ce38232

*TIME_ARGS is replaced with string.
*GstNet.NetClientClock.new , the first [0] argument (Name) can't be None as it was so i placed 'Noname' for the name.
*Since array is of GstStructure type i couldn't find len() , so i change it to string and find out the length.
*remove_probe should takes only 2 arguments.
* As i said array is of GstStructure so i also had to do this here [this]


What problems am i having

Worker log -> http://pastebin.com/Uwcs5VJe

Here are the issues:

1.

1. Traceback (most recent call last):
2.   File "/home/max/Desktop/flumotion-porting/flumotion/flumotion/common/gstreamer.py", line 51, in verbose_deep_notify_cb
3.     value = orig.get_property(pspec.name)
4. AttributeError: 'NoneType' object has no attribute 'name'

2.

1. WARN  [ 8398] "producer-video"                 feedcomponent     Jul 23 15:37:31      Could not parse pipeline: no element "None" (flumotion/component/feedcomponent.py:342)
2. WARN  [ 8398] "producer-video"                 feedcomponent     Jul 23 15:37:31      Setup failed: failure <class 'flumotion.common.errors.PipelineParseError'> at flumotion/component/feedcomponent.py:348: create_pipeline(): no element "None" (flumotion/component/component.py:586)

3.

1. Twisted traceback:
2. Traceback (most recent call last):
3.   File "/usr/lib/python2.7/dist-packages/twisted/spread/pb.py", line 896, in proto_message
4.     self._recvMessage(self.localObjectForID, requestID, objectID, message, answerRequired, netArgs, netKw)
5.   File "/usr/lib/python2.7/dist-packages/twisted/spread/pb.py", line 910, in _recvMessage
6.     netResult = object.remoteMessageReceived(self, message, netArgs, netKw)
7.   File "/home/max/Desktop/flumotion-porting/flumotion/flumotion/common/medium.py", line 279, in remoteMessageReceived
8.     self, broker, message, args, kw)
9.   File "/home/max/Desktop/flumotion-porting/flumotion/flumotion/twisted/pb.py", line 584, in remoteMessageReceived
10.     d = defer.maybeDeferred(method, *args, **kwargs)
11. --- <exception caught here> ---
12.   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 134, in maybeDeferred
13.     result = f(*args, **kw)
14.   File "/home/max/Desktop/flumotion-porting/flumotion/cache/component/a4c3f7b27db12cfd8ecdd704075aeaf1/flumotion/component/feedcomponent.py", line 249, in remote_setMasterClock
15.     return self.comp.set_master_clock(ip, port, base_time)
16.   File "/home/max/Desktop/flumotion-porting/flumotion/cache/component/a4c3f7b27db12cfd8ecdd704075aeaf1/flumotion/component/feedcomponent010.py", line 505, in set_master_clock
17.     self.pipeline.set_new_stream_time(Gst.CLOCK_TIME_NONE)
18. exceptions.AttributeError: 'NoneType' object has no attribute 'set_new_stream_time'

4.

1. Traceback (most recent call last):
2.   File "/home/max/Desktop/flumotion-porting/flumotion/cache/vorbis-component/2b18938d80c0f3474d96f6eea8f6c471/flumotion/component/encoders/vorbis/vorbis010.py", line 75, in buffer_probe
3.     in_rate = kaps[0].get_value('rate')
4.   File "/usr/lib/python2.7/dist-packages/gi/types.py", line 43, in function
5.     return info.invoke(*args, **kwargs)
6. TypeError: unknown type GstIntRange

5.

1. TypeError: event_probe_cb() takes exactly 5 arguments (4 given)

6.

1. TypeError: buffer_probe_cb() takes exactly 5 arguments (4 given)

TO DO

Fix the above mentioned problems.

PadProbeInfo' object has no attribute 'get_structure' FIXED

Here's the fix:
https://github.com/bananadine/flumotion/commit/fc6698e1274ac60aeeab5c17e867c0d14c3b693c

I asked people from the gstreamer community about these issues i'm having and for now i got an answer which helped me solve:

1. AttributeError: 'PadProbeInfo' object has no attribute 'get_structure'
   
   

I didn't see before that PadProbeInfo has a get_event method.


I asked what would be appropriate in place of TIME_ARGS which is not in GStreamer 1.0 and i'm currently waiting for an answer.

After that one i'm only left with the error relating to 'pspec' and some other minor ones.

Analyzing

1. Traceback (most recent call last):
2.   File "/home/max/Desktop/flumotion-porting/flumotion/cache/component/9d1e41022a6588631fd0de57ffb5ddee/flumotion/component/feedcomponent.py", line 965, in handle_reset_event
3.     if gstreamer.event_is_flumotion_reset(event):
4.   File "/home/max/Desktop/flumotion-porting/flumotion/flumotion/common/gstreamer.py", line 176, in event_is_flumotion_reset
5.     if event.get_structure() is None:
6. AttributeError: 'PadProbeInfo' object has no attribute 'get_structure'

Here event is a PadProbeInfo: 

https://github.com/bananadine/flumotion/blob/master/flumotion/component/feedcomponent.py#L964

proof: http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/html/GstPad.html#GstPadProbeCallback
and then it's the same type here: 

https://github.com/bananadine/flumotion/blob/master/flumotion/common/gstreamer.py#L174

but need to be GstCaps so i can do 'GstCaps.get_structure' and then on the structure use '.get_name'

Another issue:

1. Traceback (most recent call last):
2.   File "/home/max/Desktop/flumotion-porting/flumotion/flumotion/common/gstreamer.py", line 51, in verbose_deep_notify_cb
3.     value = orig.get_property(pspec.name)
4. AttributeError: 'NoneType' object has no attribute 'name'

Tried printing pspec, the result was a printed None

Where does it go wrong?

Clues (everywhere i could find pspec):

https://github.com/bananadine/flumotion/blob/master/flumotion/common/gstreamer.py#L47

https://github.com/bananadine/flumotion/blob/master/flumotion/component/feedcomponent010.py#L413
https://github.com/bananadine/flumotion/blob/master/flumotion/component/producers/videotest/videotest.py#L90
https://github.com/bananadine/flumotion/blob/master/flumotion/common/pygobject.py#L38