httpstreamer.py

As i said in my last post having localhost  or anything else instead of self.hostname is wrong.
So i changed it back and tried to fix the error i have.
With help from the flumotion team i arrived at this point .
TestStreamDataNoPlug passes but without "avatarId" which as you can see it's commented out. Here's the reason why:


track@track-HP-ProBook-4520s:~/Downloads/flumotion$ trial flumotion.test.test_component_httpstreamer

flumotion.test.test_component_httpstreamer
  TestStreamDataNoPlug
    testGetStreamData ...                                               [ERROR]
  TestStreamDataPlug
    testGetStreamData ...                                             [SKIPPED]

===============================================================================
[SKIPPED]
See #1137

flumotion.test.test_component_httpstreamer.TestStreamDataPlug.testGetStreamData
===============================================================================
[ERROR]
Traceback (most recent call last):
  File "/home/track/Downloads/flumotion/flumotion/test/test_component_httpstreamer.py", line 65, in testGetStreamData
    self.component.parseProperties({})
  File "/home/track/Downloads/flumotion/flumotion/component/common/streamer/streamer.py", line 400, in parseProperties
    self.httpauth.setRequesterId(self.config['avatarId'])
exceptions.AttributeError: 'NoneType' object has no attribute 'setRequesterId'

flumotion.test.test_component_httpstreamer.TestStreamDataNoPlug.testGetStreamData
-------------------------------------------------------------------------------
Ran 2 tests in 0.014s

FAILED (skips=1, errors=1)

And when i comment "avatarId" the test passes ( so everything else works well):


track@track-HP-ProBook-4520s:~/Downloads/flumotion$ trial flumotion.test.test_component_httpstreamer

flumotion.test.test_component_httpstreamer
  TestStreamDataNoPlug
    testGetStreamData ...                                                  [OK]
  TestStreamDataPlug
    testGetStreamData ...                                             [SKIPPED]

===============================================================================
[SKIPPED]
See #1137

flumotion.test.test_component_httpstreamer.TestStreamDataPlug.testGetStreamData
-------------------------------------------------------------------------------
Ran 2 tests in 0.013s

PASSED (skips=1, successes=1)

TO DO:
*Make httpstreamer.py work
*Configure the basic pipline for a stream i mentioned in a earlier post

Using Gtk3Reactor

Changed the PollReactor in flumotion/test/__init__.py to Gtk3Reactor
Added Gtk3Reactor to flumotion/common/testsuite.py
Made some changes to flumotion/component/common/streamer/streamer.py so flumotion/test/test_component_httpstreamer.py can pass one of two tests.
TestStreamDataNoPlug passes
TestStreamDataPlug get's skipped

track@track-HP-ProBook-4520s:~/Downloads/flumotion$ trial -r gtk3 flumotion.test.test_component_httpstreamer
flumotion.test.test_component_httpstreamer
  TestStreamDataNoPlug
    testGetStreamData ...                                                  [OK]
  TestStreamDataPlug
    testGetStreamData ...                                             [SKIPPED]

===============================================================================
[SKIPPED]
See #1137

flumotion.test.test_component_httpstreamer.TestStreamDataPlug.testGetStreamData
-------------------------------------------------------------------------------
Ran 2 tests in 0.013s

PASSED (skips=1, successes=1)

I think the skip is due to the indent left intentionally there. Don't know why tho.

TO DO:
*"localhost" in flumotion/component/common/streamer/streamer.py is wrong, need to correct that
* Find out what's with the skip above
*Try to figure out why test_common_gstreamer returns 2 fails on the Caps test

Ported the components i need for a basic stream

The porting was not much [092db60]
I almost have everything i need for a basic pipeline consisting of the following:

• Audio test and video test producers
• Audio and video encoders (Vorbis and Theora)
• Muxer (Ogg)
• Consumer (http streamer)

I have some issues when testing httpstreamer.py:

track@track-HP-ProBook-4520s:~/Downloads/flumotion$ trial flumotion.test.test_component_httpstreamer
flumotion.test.test_component_httpstreamer
  TestStreamDataNoPlug
    testGetStreamData ...                                               [ERROR]
  TestStreamDataPlug
    testGetStreamData ...                                             [SKIPPED]

===============================================================================
[SKIPPED]
See #1137

flumotion.test.test_component_httpstreamer.TestStreamDataPlug.testGetStreamData
===============================================================================
[ERROR]
Traceback (most recent call last):
  File "/home/track/Downloads/flumotion/flumotion/test/test_component_httpstreamer.py", line 64, in testGetStreamData
    streamData = self.component.getStreamData()
  File "/home/track/Downloads/flumotion/flumotion/component/common/streamer/streamer.py", line 464, in getStreamData
    'url': self.getUrl()}
  File "/home/track/Downloads/flumotion/flumotion/component/common/streamer/streamer.py", line 454, in getUrl
    return "http://%s%s%s" % (self.hostname, port_str, self.mountPoint)
exceptions.AttributeError: 'MultifdSinkStreamer' object has no attribute 'hostname'

flumotion.test.test_component_httpstreamer.TestStreamDataNoPlug.testGetStreamData
-------------------------------------------------------------------------------
Ran 2 tests in 0.012s

TO DO:
*Get successes on the test for the http streamer
*Try to figure out why test_common_gstreamer returns 2 fails on the Caps test
*Get in contact with someone from Twisted and ask for reactors compatible with gi

Ported audiotest.py, fixed __init__ in flumotion/test/

I ported feedcomponent.py and all the modules it's importing. I also ported the test for it and changed the reactor from Gtk2Reactor (which was not compatible with gi) to PollReactor.
I did the same change to the reactor in the __init__ file in the test folder and commented  a few lines.
All this lead to porting the audio test producer.
[7e20c17]

TO DO:

*Port the video test producer, audio and video encoders (Theora and Vorbis)
*Try to figure out why test_common_gstreamer returns 2 fails on the Caps test
*The goal is to make a simple stream pipeline work under gstreamer 1.0

Things i did today

Ported check.py [1cd8163]
Well, that was nothing really, just search and replace.
Had some trouble with the tests , every one of them returned SKIPPED, so I deleted the repo from my pc . I figured that there was trouble with my repo on bananadine/flumotion.git which had an older version of flumotion and i've pulled from there on my local repo. (didn't really know how to work with git when i was setting up so i messed something up then) Now that's fine and i learned how to use git (at least basics for now).

The test_common_gstreamer test still returns me 4 successful and 2 fails. I gave the code to a mentor (mithro) and it run just fine there. Still don't know what's the trouble. All my gstreamer packages are up to date.

TO DO:
*Try to figure out why test_common_gstreamer returns 2 fails on the Caps test
*Port feedcomponent.py
*Resolve the issue with gtk2reactor and gi (my earlier post where i commented everything in the test/__init__.py  so i can run test_common_gstreamer)
In twisted.internet.gtk2reactor i found this piece of code

# System Imports

import sys



if 'gi' in sys.modules:

    raise ImportError(

        "Introspected and static glib/gtk bindings must not be mixed; can't "

        "import gtk2reactor since gi module is already imported.")



# Disable gi imports to avoid potential problems.

sys.modules['gi'] = None

Hmmm

gstreamer.py and test_common_gstreamer.py test error (Part 2)

The error in the test that was telling me something is using "import gobject" is gone. It was due the __init__ in the flumotion/test/ folder, it was importing Gtk2Reactor which in turn is importing gobject. I commented everything in it and after some other tweaking in gstreamer.py and test_common_gstreamer.py the test (almost) passes:

flumotion.test.test_common_gstreamer
  Caps
    testCaps ...                                                         [FAIL]
    testCapsStreamheader ...                                             [FAIL]

DeepNotify
    testDeepNotify ...                                                     [OK]
  Factory
    testFakeSrc ...                                                        [OK]
  TestProperty
    testHasProperty ...                                                    [OK]
    testHasPropertyValue ...                                               [OK]
===============================================================================
[FAIL]
Traceback (most recent call last):
  File "/home/track/flumotion/flumotion/test/test_common_gstreamer.py", line 42, in testCaps
    'video/x-raw, width=(int)10, '
  File "/usr/lib/python2.7/dist-packages/twisted/trial/unittest.py", line 270, in assertEqual
    % (msg, pformat(first), pformat(second)))
twisted.trial.unittest.FailTest: not equal:
a = '<GstCaps at 0x2ed5a30>'
b = 'video/x-raw, width=(int)10, framerate=(double)5; video/x-raw, width=(int)15, framerate=(double)10'


flumotion.test.test_common_gstreamer.Caps.testCaps
===============================================================================
[FAIL]
Traceback (most recent call last):
  File "/home/track/flumotion/flumotion/test/test_common_gstreamer.py", line 49, in testCapsStreamheader
    'streamheader=<...>')
  File "/usr/lib/python2.7/dist-packages/twisted/trial/unittest.py", line 270, in assertEqual
    % (msg, pformat(first), pformat(second)))
twisted.trial.unittest.FailTest: not equal:
a = '<GstCaps at 0x2ed5c50>'
b = 'streamheader=<...>'


flumotion.test.test_common_gstreamer.Caps.testCapsStreamheader
-------------------------------------------------------------------------------
Ran 6 tests in 0.012s

FAILED (failures=2, successes=4)


link to commit [dbf3f98]

TO DO:
*Pass the whole test_common_gstreamer
*Continue the porting with feedcomponent.py and check.py

gstreamer.py and test_common_gstreamer.py test error

The diff of gstreamer.py: http://pastebin.com/U4PNC2Vj
The diff of test_common_gstreamer.py: http://pastebin.com/e9W5RkPA

When i run "trial flumotion.test.test_common_gstreamer" i still get:

flumotion
  test
    test_common_gstreamer ...                                           [ERROR]

===============================================================================
[ERROR]
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/twisted/trial/runner.py", line 660, in loadByNames
    things.append(self.findByName(name))
  File "/usr/lib/python2.7/dist-packages/twisted/trial/runner.py", line 470, in findByName
    return reflect.namedAny(name)
  File "/usr/lib/python2.7/dist-packages/twisted/python/reflect.py", line 464, in namedAny
    topLevelPackage = _importAndCheckStack(trialname)
  File "/flumotion/flumotion/test/test_common_gstreamer.py", line 18, in <module>
    import gi
  File "/usr/lib/python2.7/dist-packages/gi/__init__.py", line 23, in <module>
    from ._gi import _API, Repository
exceptions.ImportError: could not import gobject (error was: ImportError('When using gi.repository you must not import static modules like "gobject". Please change all occurrences of "import gobject" to "from gi.repository import GObject".',))

flumotion.test.test_common_gstreamer

Also i learned how to use github properly and from now i'll post links to the commits.

Porting gstreamer.py and test_common_gstreamer.py

To make a port for the audio test  i'll have to port gstreamer.py, feedcomponent.py and check.py , because audiotest.py imports them.
(gst.* is replaced with Gst.* everywhere)
1. I'll start off with gstreamer.py which is a set of  gstreamer helpers:

  import gst

#is replaced with

  import gi
  gi.require_version('Gst', '1.0')
  from gi.repository import Gst

2. In function element_factory_has_property:


    e = gst.element_factory_make(element_factory)

#is replaced with

    e = Gst.element_factory.make(element_factory)

3. In function element_factory_has_property_value:

    e = gst.element_factory_make(element_factory)

#is replaced with (same as the one before)

    e = Gst.element_factory.make(element_factory)

4, In function element_factory_exists:

    registry = gst.registry_get_default()

#is replaced with

    registry = Gst.registry_get()

5. In function get_plugin_version:

    plugin = gst.registry_get_default().find_plugin(plugin_name)

#is replaced with

    plugin = Gst.registry_get().find_plugin(plugin_name)

6. In function get_state_change:

    table = {(gst.STATE_NULL, gst.STATE_READY):

             gst.STATE_CHANGE_NULL_TO_READY,

             (gst.STATE_READY, gst.STATE_PAUSED):

             gst.STATE_CHANGE_READY_TO_PAUSED,

             (gst.STATE_PAUSED, gst.STATE_PLAYING):

             gst.STATE_CHANGE_PAUSED_TO_PLAYING,

             (gst.STATE_PLAYING, gst.STATE_PAUSED):

             gst.STATE_CHANGE_PLAYING_TO_PAUSED,

             (gst.STATE_PAUSED, gst.STATE_READY):

             gst.STATE_CHANGE_PAUSED_TO_READY,

             (gst.STATE_READY, gst.STATE_NULL):

             gst.STATE_CHANGE_READY_TO_NULL}

#is replaced with

table = {(Gst.STATE.NULL, Gst.STATE.READY):

             Gst.STATE.CHANGE_NULL_TO_READY,

             (Gst.STATE.READY, Gst.STATE.PAUSED):

             Gst.STATE.CHANGE_READY_TO_PAUSED,

             (Gst.STATE.PAUSED, Gst.STATE.PLAYING):

             Gst.STATE.CHANGE_PAUSED_TO_PLAYING,

             (Gst.STATE.PLAYING, Gst.STATE.PAUSED):

             Gst.STATE.CHANGE_PLAYING_TO_PAUSED,

             (Gst.STATE.PAUSED, Gst.STATE.READY):

             Gst.STATE.CHANGE_PAUSED_TO_READY,

             (Gst.STATE.READY, Gst.STATE.NULL):

             Gst.STATE.CHANGE_READY_TO_NULL}

7. In function have_error:

        changes = [gst.STATE_CHANGE_NULL_TO_READY,

                   gst.STATE_CHANGE_READY_TO_PAUSED,

                   gst.STATE_CHANGE_PAUSED_TO_PLAYING]

        extras = ((gst.STATE_PAUSED, gst.STATE_CHANGE_PLAYING_TO_PAUSED),

                  (gst.STATE_READY, gst.STATE_CHANGE_PAUSED_TO_READY),

                  (gst.STATE_NULL, gst.STATE_CHANGE_READY_TO_NULL))

#is replaced with

        changes = [Gst.STATE.CHANGE_NULL_TO_READY,

                   Gst.STATE.CHANGE_READY_TO_PAUSED,

                   Gst.STATE.CHANGE_PAUSED_TO_PLAYING]

        extras = ((Gst.STATE.PAUSED, Gst.STATE.CHANGE_PLAYING_TO_PAUSED),

                  (Gst.STATE.READY, Gst.STATE.CHANGE_PAUSED_TO_READY),

                  (Gst.STATE.NULL, Gst.STATE.CHANGE_READY_TO_NULL))

Now i'll have to port the unit test for the gstreamer module, test_common_gstreamer:

1.

  import gst

#is replaced with

  import gi
  gi.require_version('Gst', '1.0')
  from gi.repository import Gst

2. In class Caps, function testCaps:


caps = gst.caps_from_string(
            'video/x-raw-yuv,width=10,framerate=5.0;video/x-raw-rgb,'
            'width=15,framerate=10.0')
        self.assertEquals(gstreamer.caps_repr(caps),
            'video/x-raw-yuv, width=(int)10, '
                          'framerate=(double)5; video/x-raw-rgb, '
                          'width=(int)15, framerate=(double)10')



#is replaced with


    def testCaps(self):

        caps = Gst.caps_from_string(
            'video/x-raw-yuv,width=10,framerate=5.0;video/x-raw-rgb,'
            'width=15,framerate=10.0')
        self.assertEquals(gstreamer.caps_repr(caps),
            'video/x-raw-yuv, width=(int)10, '
                          'framerate=(double)5; video/x-raw-rgb, '
                          'width=(int)15, framerate=(double)10')


3. In class FakeComponent, function run_it_a_little_while:


    p.set_state(gst.STATE_PLAYING)
    m = p.get_bus().poll(gst.MESSAGE_EOS, -1)
    p.set_state(gst.STATE_NULL)


#is replaced with

    p.set_state(Gst.STATE.PLAYING)

    m = p.get_bus().poll(Gst.MESSAGE_EOS, -1)

    p.set_state(Gst.STATE.NULL)

After the porting i tried to run the test using Trial and i got:


[ERROR]
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/twisted/trial/runner.py", line 660, in loadByNames
    things.append(self.findByName(name))
  File "/usr/lib/python2.7/dist-packages/twisted/trial/runner.py", line 470, in findByName
    return reflect.namedAny(name)
  File "/usr/lib/python2.7/dist-packages/twisted/python/reflect.py", line 464, in namedAny
    topLevelPackage = _importAndCheckStack(trialname)
  File "/flumotion/flumotion/test/test_common_gstreamer.py", line 18, in <module>
    import gi
  File "/usr/lib/python2.7/dist-packages/gi/__init__.py", line 23, in <module>
    from ._gi import _API, Repository
exceptions.ImportError: could not import gobject (error was: ImportError('When using gi.repository you must not import static modules like "gobject". Please change all occurrences of "import gobject" to "from gi.repository import GObject".',))

flumotion.test.test_common_gstreamer

Even tho i've changed all occurrences of "import gobject" to "from gi.repository import GObject"

First steps

Installed the packages i'll be needing

 sudo apt-get install python-gi python3-gi \
     gstreamer1.0-tools \
     gir1.2-gstreamer-1.0 \
     gir1.2-gst-plugins-base-1.0 \
     gstreamer1.0-plugins-good \
     gstreamer1.0-plugins-ugly \
     gstreamer1.0-plugins-bad \
     gstreamer1.0-libav

In this week i'm going to port some modules which i can use 

to set up a basic configuration of flumotion for a stream pipeline. 

I'm going to use the audio test and video test producer.

To import the new PyGst library i do this:

 import gi
 gi.require_version('Gst', '1.0')
 from gi.repository import Gst

Intro

My name is Max and in this blog i'll post daily progress of the porting of Flumotion  from gstreamer 0.10 to gstreamer 1.0.